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Introduction 



Frank Caplan: 

My name is Frank Caplan, and I am chairman of the Long 
island Section of the American Society for Quality Control and 
co-chairman of the seminar. The other co-chairman, Mr. LeRoy 
Walser, of the Department of Education, will be speaking to you 

in a few minutes. 

I would like to give a little background for this 
session as well as I can at this stage. Because of the almost 
total lack of knowledge of the critical nature of how to achieve 
quality on the part of our national work force at all levels of 
business, industry, government, and institutions, and because of 
the resultant waste of money, time, limited natural resources, 
and because of our declining competitive position in domestic 
and world market places, there is a need to incorporate 
appropriate portions of the Quality Sciences into every course 
taken in every school, from kindergarten through graduate 
school, and in continuing education. 

The first extensive speaker this morning, Bill Golomski, 
has reminded me there are some states the t have formal 
pre-school programs which should have Quality Sciences started 
there. 

I have been working on this concept and attempting to 
obtain participation by state and national organizations. The 
chap who got me involved in this at the beginning unfortunately 
could not be with us today. His name is Mr. Stanley Seifer. He 
is Director of the Long Island Quality Institute, which is the 
educational arm of the Long Island Section of the ASQC. His 
wife is ill, and he was unable to get away, but I wanted you to 
understand that while I have been much in the forefront of this 
effort for the last six years, this was all originally Stanley's 
idea. He didn't have to sell me very hard. 

We have three principal speakers today, Mr. Golomski, 
Dr. Godfrey, and Dr. Basil i. Each of you have or have available 
to you a biographical sketch of each of these gentlemen and so, 
in the interest of time and getting as much of the message 
across as we can, we will forego any extensive introduction of 
these individuals, but I think it is important that you should 
recognize that all of them, directly, immediately, and totally 
or to a significant degree at least (as well as myself and other 
speakers from the AjQC that you may hear today or see at the 
breaks and at lunch) are involved in education. 

The American Society for Quality Control was founded in 
1946 as the outgrowth of War Production Board courses that were 
given during World War II by college professors who were 
involved particularly in the area of applied statistics in 
industry because the War production Board was confident the 
American effort in World War II had to be supported by all 
possible tools and recognized that this was an extremely 
valuable one. Groups formed as a result of these courses and 
they ultimately joined together in 1946 to form ASQC. 
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The point I want to make with this very brief history of 
an outstanding organization is that it was founded as a result 
of educational effort, it has been dedicated since its inception 
to education in all areas associated with the subject of 
quality, and in the last eight years has come to considerable 
prominence in the area of quality worldwide. 

So we are not unaware of the educational concerns and we 
are fully convinced that the only solution to our problem for 
the long term, our whole societal problem, is the question of 
quality. So we would like to encourage you to listen to what we 
have to say in that regard. 

At this stage I would like to turn the meeting over to 
my co-chairman, Mr. LeRoy Walser, Executive Director of the 
Federal Interagency Committee on Education. LeRoy. 



Introduction 

LeRoy Walser: 

I also would like to welcome you to this important 
meeting. ASQC has dedicated itself to the pursuit of quality, 
and education seems always to be pursuing excellence. This 
meeting, we hope, will set up the groundwork for considering, 
nationally, issues of qualify that affect education. 

Now, I have the distinct privilege and opportunity of 
introducing to you today Dr. Peter Greer, who is the Deputy 
Under Secretary for the Office of Intergovernmental and 
Interagency Affairs in the Department of Education. It has been 
a real opportunity to work with Dr. Greer, an insightful man, a 
man who has a lot of motion and direction to him, and it has 
been a pleasure to work with him. 

Without giving an awful lot of background about Dr. 
Greer, who has been involved in education professionally, I 
guess all of his life, as superintendent of schools, as a 
director of national projects before he came to the Department 
of Education, and now is a very influential and hardworking man 
whom I respect. Dr. Greer. 



Welcome 

Dr. Peter R. Greer: 

That is one of the nicest introductions I have received 
in a while. I also can caddy. I was a Triple A caddy at the 
Broadmoor Golf Course in Colorado Springs. Other than that, I 
can't do another thing. 

This is a cameo appearance not because my time is so 
valuable and yours is not, but because I just returned from 
Montevideo, Uruguay. I was there a week working with eleven 
nations on the drug problem, so I am unable to do my job and 
spend an entire day on this important topic. But, LeRoy Walser 
insisted that I be here to open the session, and I always do 
what LeRoy tells me to do. 
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I am therefore the loser, and you are the smart ones. 
In other words, you have the. good instincts that this topic may 
be an important one, that the quality sciences have implications 
for education. You may be pioneers today. All of us go to 
conferences, day after day, or week after week. Some of them 
really don't make a difference, but it is important that we be 
there. This one may make a difference. So I think you will 
find this exciting. 

In looking at the the pre-registrations, there seems to 
be a good diversity of groups, and this is vintage FICE. FICE, 
as you know, the Federal Interagency Committee on Education, has 
the job of bringing together various agencies, both federal and 
non-federal. I noticed on the pre-registration list that we 
have both, at least in the pre-registration. I don't know most 
of you. I am guessing that if you pre-registered, you showed, 
arof again, congratulations for being pioneers. Hopefully at 
lunch you will talk to each other about this topic, and you will 
meet new people, and a lot will come out of that. 

Now, why is this topic potentially important? I wasn't 
going to say, this topic is not important, I don't know why you 
are here. I wasn't going to say that, but what is my argument 
that this is an important topic? 

Well, first of all, quality in education is missing. We 
all know that. It is missing in general. I mean, a few years 
ago someone named Fenwick English came up with the idea of 
mapping out what you are teaching in the classroom and in the 
curriculum; people went crazy. He picks up about $2000 a speech 
helping schools to map out what they are doing. We are in 
trouble in terirs of quality in education. 

When we find out that the testing — when a West Virginia 
doctor finds out that the testing programs in most schools are a 
little faulty, and that everyone is above average, we are in 
trouble. We are in trouble in some way and quality is needed. 

When the United States spends $310 billion a year on 
education, 7 percent from the federal government, 50 percent 
from the states, and 43 percent from the local taxpayers, and 
yet we come out last or next to last or in the middle of the 
pack in biology, physics, chemistry, and most other indicators, 
when China and Japan— when we stay with China and Japan until 
about the fifth grade, and then they just move right past us, 
maybe that is a question of quality. 

When the public is demanding quality, when they are 
saying, we will pay more but we want to see some results, then 
the idea comes in that perhaps quality control, quality 
sciences, are really important. When teachers are confused and 
everyone is complaining about top down, bottom up, middle out, 
and people are spinning around, maybe this topic is really 
important, and ray argument would be that. 

You have the reform movement, which makes this topic a 
very important one. Secretary Bennett, as you know, presented 
his homework assignment to President Reagan on April 26th, what 
has happened in the last five years, and the result is that we 
have a reform movement, but we don't see the results. 



So again, I think it is timely to talk about this topic. 
If LeRoy had put this idea together with the other gentlemen 
prior to "A Nation At Risk", or even three years ago, perhaps it 
wouldn't be timely. It is timely right now. 

Why? Accountability and assessment are buzzwords, but 
important words, in terms of what is going on in education. You 
have such things as state report ca^ds, assessing schools 
against one another on a series of criteria. You have community 
report cards looking at such things as student attendance, 
teacher attendance, dropout rates, test scores, things of that 
sort. You have the career ladders and merit. You have to have 
criteria, and that is the big thing. When teachers say I don't 
want anything to do with merit plans or career ladders, it is 
because of the criteria and who is doing the evaluation, In 
other words, the quality evaluations are really important. 

You have evaluation of personnel. Principals have to be 
evaluated better. Superintendents have to be evaluated better. 
Teachers have to be evaluated better, and with fairness, and 
when you say fairness, you are not talking about some mystical 
term, you are talking about criteria. You are talking about 
quality criteria. 

You have a National Teachers' Board, 63 people who will 
be struggling to come up with ways to transfer quality criteria 
and quality teachers and the certification across state linen. 

You have the Association of School Business Officials 
trying to come up with certification, with quality control 
within their own ranks. You have superintendents and principals 
trying to come up with quality control in their own ranks. 

Why? Well, probably because you have a National 
Teachers' Board, and they don't want to be left behind so that 
only teachers are seen as professional. You even have such 
quality control and quality criteria needs as those found in New 
Jersey. They will probably be taking over a school system. This 
will be the first time, perhaps in the history of the United 
States, where a state has said, you are so bad school system, in 
this case Jersey City, that we are going to take you over. And 
what happens when there is a takeover, based on criteria, based 
on evaluation, based on lack of quality? They fire the 
superintendent, they fire the superintendent's associates, and 
they fire the school board. 

So, to do something like that you have got to be sure 
about what you are doing, if this Jersey City takeover works, 
there are four other states that have the same thing on the 
books, Arkansas, South Carolina, and I can't remember the other 
two. We want to have quality in this, too, so there is 
fairness, so that there is fairness. 

It is one thing to be toughminded. It is one thing to 
be bold. But you have to be fair also, and this is where 
quality comes in. 

Then you have technology. There is the whole area of 
technology in terms of the software. Technology has an impact 
on equity, on all students being able to perform in the basics. 
It has done more than other attempts in terms of teacher 
attendance, student attendance, fewer drop-outs, and kids really 
learning. I mean, the whole area of technology cries out for 
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quality. You also have business practices. Can we adapt the 
concept of business practices to school systems? Right now you 
know that there is great talk about the local level, the small 
teams as you would find, I guess, making Volvos or something. I 
never had a car that cost more than $6,000, so I don't know what 
I am talking about when I talk about Volvos. All of my cars are 
like my Corvair where I shifted it and I didn't know if I would 
go forward or backward, which has made me a very tense person, 
and emotional. 

But the business practices of the small groups, the 
small teams working, this seems to be a fertile area for 
quality, for the quality sciences. The whole idea of drugs and 
the materials to prevent drug abuse, and the curricula that can 
be used, is a fertile ground. I mean, everyone has something to 
help cure drugs now. When I first came aboard here in 
Washington, in the Department of Education, I met with a 
drug-sniffing dog company from Oklahoma, or representatives of 
that company. Everyone—but how do you choose what is good and 
what is bad? What is the criteria even for that? 

In the famous book from the Department of Education, 
Schools Without Drugs , page 26 gives you a start on how to 
ferret out what is good and what is bad, but it is not enough. 
It is not enough. So here is another potential area. 

The whole idea of sharing what works well requires 
crite <a to be used. School people like me were famous in the 
sixties and seventies for taking on garbage. Garbage. Anything 
looked good. Here is a textbook on the humanities called The 
Humanities and You . It looks good to me. I don't even have to 
look at the table of contents. And everyone knows there is a 
great controversy on textbooks in general, and that California 
is trying to do something about it, and leading the league in 
that. But I mean, even in the examination of textbooks there is 
a need for the quality sciences. 

And how about the whole idea of choice? The entire 
state of Minnesota now is saying, parents, you can choose any 
school you want in the entire state, and other states have toyed 
with that, and two or three have defeated it, such as Colorado. 
But Minnesota is leading the league in that, and other states 
are going to do it. 

Well, what is the criteria for choosing a school? I 
think that's a fertile area right there. On what basis do you 
choose a school? 

And then finally I would guess that Secretary Bennett, 
were he here, would say there is no mystery to a lot of the 
things that go on in education. Educators try to make things a 
mystery. Researchers talk to each other. But I think with the 
use of quality sciences, we can prove that in many ways there is 
no mystery to what works in education and how we can have a 
quality education. 

So what I am trying to say is, with all these examples, 
that statistical process control and quality cost and total 
quality control and failure analysis and zero defects should 
take on our deep interest all day today. There is an ally out 
there, I think, called the quality sciences, and maybe this 
ally, and maybe you through your discussions can show that the 



9 

ERJC 



9 

10 



quality sciences can be an ally to education, and can help 
reform education and bring us the educational system that we 
want for our children. 

And finally , I guess, by attending this meeting, what 
you are doing is that you are saying, let's see if we can will 
the means to make education better through the quality sciences. 
I am fond of telling a story about my poor son. The moment he 
finds out I am telling this story he is going to kill me. But 
when he was in Little League, back in Maine, he used to strike 
out all the time, and he used to strike out all the time in the 
last of the sixth with men on base, usually with the winning run 
on third base. And the parents would go crazy, and they would 
call him names, and they would ask who spawned this kid. They 
were really upset. 

And my son would say, gee, Dad, I keep striking out. I 
said, I know, I know. And he'd say, I wish I could hit better. 
And I said to him, I'll give up my job, I'll throw 100, 200 
balls a night, you'll hit, you'll hit. And he said, I don't 
want to practice, I just want to hit. 

He hadn't realized the difference between wishing and 
willing, and what you are doing here at this meeting is willing 
the means of having higher quality education in the United 
States through the quality sciences, not just wishing the whole 
thing. 

I am going to read the minutes of this meeting very 
carefully. A lot of paper is put in front of me. I don't 
always read it. But I am going to be very interested in your 
conclusions and the results of your conversations. I think that 
there is a potential, LeRoy, for articles in the national 
education magazines and others that you all would know about 
that I don't. 

You are going to be listening to a lot of presentations. 
I hope you ask questions of the presenters. You will be in a 
wrap-up session, and in this wrap-up session you are supposed to 
determine the level of interest in the topic of quality as it 
applies to education. You are supposed to consider ways to 
proceed afterwards. 

In other words, once you leave the door hopefully you 
won't say, well, that is over, what is the meeting tomorrow or 
what are the meetings after the Labor Day holidays? What is the 
follow-up to a meeting of this sort? Will the ways. Don't wish 
after the meeting. 

Mr. Spencer Hutchens and Dr. Kay Wright and Ms. Virginia 
Robinson are going to be leading the closing oiscussion, and I 
hope you come to that with some really first-rate ideas. 

So, it is good to be with you. I apologize that I am 
leaving. I gave you my excuse. I am envious in a sense. 
Especially after I wrote this speech, I said, you know, this is 
a good conference. 

Maybe I should have rescheduled — maybe I shouldn't be 
doing the things I am doing for the rest of the week. But that 
is, again, why I am going to be listening and watching for the 
results. So, good luck, and have a good day. 

Thank you. 
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Mr. Walser: 



Thank you very much, Dr. Greer. We appreciate that 
introduction. 

A couple of little items. In your registration packet 
you saw this form for questions to the presenters. I think each 
of you, if yours is like my packet, have three of them. 
However, you are not limited to three. I would like to 
introduce George Spicely and Diane Ruffner, who will be runners 
to gather these questions that you have and to provide you with 
additional forms, if you need them. I would recommend to you 
that at this kind of conference and this kind of presentation, 
it is very useful for the presenters to receive your questions 
because it gives the presenters another opportunity to talk 
about things that are important to them. 

It is extremely useful to hear from you the kinds of 
questions, the kind of observations, and the points of possible 
disagreement. If you would write those down when they occur to 
you so you don't have to say to yourselves, what was that 
thought that I had and how does it fit? You can hand them in 
right then by merely raising your hand like this and holding the 
form up, and one of the two runners will come by and pick it up 
and give you another one. 

This is going to be a factory production of high quality 
reaction. And we'd appreciate that kind of interaction from you 
as the day proceeds. If you have some questions for Dr. Greer, 
just put Dr. Greer, your remarks about such and such were right 
on target, or they are off, or whatever you wish to say. Let it 
flow as we work through this conference. We would appreciate 
that. And if you have any other questions or any other 
concerns, please let us know. The FICE staff members are here 
to support you and your attendance today. 

I would like now to turn to Mr. Bill Golomski, who will 
give his presentation. We will not take a lot of time to talk 
to you about his qualifications, his biography. It is included 
in your packet. Please look at it. Please read it. I hope you 
have read what we call maybe a little teaser in terms of what he 
will be presenting and talking about. And he is anxious, and he 
is here, so Mr. Golomski. 
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Presentation: Broad Overview of Quality 



Mr. Golomski: 

Thank you very much, LeRoy. Good morning, everyone. It 
is good to be here with you. I am always delighted to meet with 
a group of people who are in education and those who are in 
other parts of government. This is because I have a great 
respect for the kinds of things you are doing and under the 
conditions that you are operating. 

I am going to talk about two things today. The first 
has to do with how to look at our work from the perspective of 
the quality sciences. This is, to what extent can the operation 
of our various organizations, agencies, departments, and so on 
be improved using some of these concepts? 

Then, the second has to do with how there might be some 
concepts here that could fit into the educational program 
starting with pre-school and working its way upward. 

And by the way, even though there will be ample time for 
discussion in the panel, if you have any question along the way 
that you would like to ask because it is distracting you from 
what I might say afterwards, simply raise your hand and I will 
try and answer it at that time. 

Now, one of the interesting things is that no matter 
what the educational level of a person is, there are 
opportunities for employing concepts of quality improvement. 
Last night when I checked into the hotel I noticed that a 
convention of anesthesiologists is meeting here in Washington, 
and they are going to be talking about a variety of interesting 
things, I am sure. Very coincidentally , when I was in Orlando 
earlier this week there was a convention of office managers of 
anesthesiology practices. It is interesting that the two groups 
are meeting in different cities. Were I to design a conference, 
I would hope that somehow those who are in charge of managing 
the business offices might meet with their professional 
counterparts some day. But, perhaps, the gaps in the 
organization chart or the educational backgrounds are so great 
that they just don't think it's possible for them to meet 
together. 

But anyway, let me tell you my story about 
anesthesiologists. I was in for my second annual hernia 
operation a few years ago. With the first one, I had 
investigated the surgeons carefully and found a fellow whc had a 
marvelous record. So, I decided he is the man who is going to 
take care of me. What I didn't realize was that he was within 
six months of retirement; but had I known that I don't think it 
would have made any difference. 

Well, anyway, into the surgery and out into the cardiac 
care center, and so it turned out that I was allergic to a 
number of the anesthetics. That hadn't been checked out the 
night before, even though I had told them and they supposedly 
made the right selection. The surgery wasn't done well; about 
60 percent of the final operations done by that MD before he 
retired were questionable. 
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Again, just because someone has done great things in the 
past is no indication they will in the future. That is one of 
the things that is very important in the qrslity sciences. That 
is, the past is not a sure predictor of th future. The past 
might be of some help, but we also have to ook at some 
prevention systems. 

Now, the interesting th: : ng was, when I went for my 
second hernia operation, I was in my room the night before 
surgery. A gentleman came in with a tray full of 
pharmaceuticals, and he said, "I am here to give you your shot." 
I said, "That is rather interesting. I am going into surgery 
t morrow morning. I don't know why I need a shot tonight." 

He said, "Don't worry, I give these shots without any 
pain." I said, "That's not the point. I think you are 
mistaken." And he said, "Well, a lot of people are unhappy 
about hypodermic needles and all that, but I do this 
professionally. " 

I r,aid, "Look, I'll make a deal with you. You look on 
your chart and tell me what my name is, and then come over and 
look at my bracelet, and let's see if we can find out whether or 
not the two match." So he announced my name: "Chung Chu Lee." 
I said, "That's a nice name. It doesn't happen to be mine." I 
said, "Why don't you come over and look at my name?" Well, he 
walked toward me with the hypodermic needle behind his back. I 
said, "Look, this is a fair game. I am serious." So he put it 
down, came over, and saw my name. He says, "Oh, I must be in 
the wrong room, or maybe the patient checked out, or maybe I'm 
on the wrong floor." And I talked to him, "Look," I said, 
"don't be so intent on getting rid of all these drugs on your 
tray, following your schedule. There is a quality aspect to 
your job as well." 

Well, the next morning I was wheeled down into surgery. 
While I was waiting in the waiting room the anesthesiologist 
came over to prepare me. She suid, "Well, I am going to prepare 
you for the surgery." I said, "Now, the last time I was in here 
I had serious problems. Make sure you have the right 
anesthetic. " 

And she said to me, "Well, didn't an anesthesiologist 
talk to you last night?" I said, "No!" She said, "Well, they 
are supposed to. You have to be in surgery in five minutes." I 
said, "No, I can be a streaker and run out of this hospital. I 
don't have to be in surgery." 

The interesting thing is, they were more interested in 
schedule than quality. They had a certain routine of things 
that had to be done, and I was becoming an obstacle to the 
routine as they had established it. Even though I was trying to 
tell them all the things to look at, they filtered out all of 
the signals. These are the kinds of things that can happen even 
in systems where you have very bright people. They lose track 
of what they are really up to. It is the schedule or cost that 
becomes most important. They lose perspective that there is a 
quality aspect of what they do. 

We are not talking about people at the bottom of the 
organization chart. We are talking about people who are up 
the top of the organization in many cases. By the way, goi»~ 
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back to ray experience this week, the anesthesiologists here, the 
managers in Orlando; I should also tell you that I was working 
with a group of people in medical supply who furnish the 
anesthetics to the anesthesiologists. They were having their 
convention in a third place. I wonder if these three groups 
ever talk to each other. 

Try and find out who are the suppliers of information, 
who are the people with whom you deal, who are those who are 
part of the system, and what is their role? 

Dr. Greer talked about the firm from Oklahoma who wanted 
to get him interested in the drug-sniffing dogs. What is the 
problem with that? It is detection of the problem, not 
prevention. So from the quality sciences point of view, we are 
not saying detection is bad, we are saying that if you can 
develop a system that is associated with prevention of a 
problem* than that is a superior system to one that is concerned 
with detecting all sorts of errors. 

One of the little surveys that I run with just about 
every group that I talk to is to find out whether there were any 
errors in your last paychecks. By the way, any errors that 
anyone here had in their last paycheck? None? Now, the 
question is, how did that happen? Are there Japanese in the 
payroll department? How could it happen that Americans did that 
job so well that not a single person here raised their hand? 
And why does that system work? 

Number One; there is instantaneous feedback. Number 
Two; you want that system to work. You don't want to blame 
anyone. You want the system to work. See? Now, why can't we 
have that attitude toward all our systems? Quick feedback, no 
retribution; trying hard to make the system to work, rather than 
concentrating on fixing the blame. 

That is an important attitudinal element of anything 
that has to do with quality. 

Oftentimes when we consider doing something in a new 
way, one of the first things we talk about is to make a needs 
analysis. We find out whether or not those with whom we work 
have a need for whatever service or program we are trying to 
start. That is okay in many cases, but it is not good enough. 
Let me tell you why a needs analysis is good in some cases and 
not in others. 

First of all, we want to be responsive to citizens, to 
people in other departments with whom we work, to our customers, 
to our employees, and even for working well with those who 
supply us information or whatever. But when there are new 
concepts around, like concepts of the quality sciences, a needs 
analysis might not identify all the elements and all the 
programs and all the systems that could be used to improve 
quality. So, there are times when you need an expert who comes 
in from outside the organization to bring in ideas and to look 
at problems in a way that you never thought of. 

And so, I have nothing against needs analysis, but I am 
saying in many cases needs analysis is not enough. It is not 
enough. 




Let me give you an example. There is a story that was 
told during World War II about a group of Congressmen trying to 
decide what to do about shells exploding in the hands of 
soldiers before the shells got into artillery pieces. And so a 
group of people in a hearing got together, and there were 
various suggestions. One was, let's shoot off every other shell 
to see whether the adjacent ones were okay. The next most 
senior person on the committee, not to be left out, said, well, 
we'd lose too many shells that way, why don't we shoot out every 
fourth one? 

And then the next person, maybe every eighth one, and so 
on, until all that was left to the junior member of the 
committee was, one to the two whatever power, perhaps fifteenth 
power. Then they discovered that a legal education is of no 
help in technical areas. It might be good for negotiation; it 
might be good for showing your constituents that you are 
involved. They had no expertise. There was no technology. 
There was no common sense. It was just an attempt to try and 
solve the problem. 

So there are cases even when you have bright people who 
are trying to solve problems that you have to find the 
appropriate expert to try and help solve some of these problems, 
and sometimes they are hard to find. 

Leadership is extremely important also, and in many 
cases, if you are lucky, the leader can bring in either new 
ideas or at least make the organization receptive so that it is 
willing to consider change. And that is very important in the 
quality area. 

Usually when we try to learn how to change an 
organization we might have certain traditional sources with whom 
we work. Those of you at the federal level might work with 
those in other federal agencies. You might work with those at 
the state level, municipal level, and so on. And that is the 
' domain within which you work. 

I was reminded of a potential client who called me one 
time and said, "Do you know anything about flea collars and tick 
collars?" I said, "No; what is your problem?" He said, "Well, 
these boxes of flea collars and tick collars get to the 
supermarket and occasionally there is nothing in them." And I 
said, "Well, where does this happen?" He said, "We don't know 
if they are stolen at the supermarket, or if they are not 
inserted in the factory, but it is a very serious problem to our 
industry. " 

I said, "It has nothing to do with your industry. Any 
parent who has tried to assemble toys on Christmas Eve and has a 
little bag of nuts and bolts and screws occasionally discovers 
that the bag is incomplete. So all of you are talking about 
here is a problem of why isn't a package complete in the sense 
of all the things that are supposed to be in it." 

H» says, "Oh, you don't understand our industry." He 
said, "This flea collar and tick collar industry is different." 
Well, the whole point was, he couldn't get himself out of this 
trap that he had blocked himself in. All he could think about 
was how specific this problem was to his company; how it was 
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so different from any other company that had it. It was 
unlikely that anyone could come in there and come up with a 
solution. 

Now, that might be true in some problems, but in many 
cases the best ideas for any organization come from outside the 
profession or outside the industry or even outside the country. 
That is one of the interesting things about how we bring change 
into organizations. I was told a few blocks away from here we 
have a Statistical Department at the U.S. Department of 
Education. I would be surprised if all of the statistical 
methods used there were developed by people in the educational 
field. I will bet some of the methods that are used there were 
developed at Guiness Brewery years ago, when the t test was 
developed there, I will bet some of the other tests that are 
used there were developed in agriculture, and in manufacturing, 
and in research and development, and in communications. 

So, one of the interesting things is that departments 
that are truly effective have learned how to bring in ideas from 
other places rather than simply to read their own journals and 
talk with people within their own disciplines. That is one of 
the reasons we are here today; to simply try and get us to think 
about how we can learn from other disciplines. 

Another important concept in the whole quality area is, 
we like to get people to understand that when we assign them to 
a job or when we hire them, that they really have two jobs. The 
first is to do whatever your job description says you are 
supposed to do. And then the second is to improve on it! And 
in some cases the improvement might even result in the 
elimination of that job. 

Let me give you an example. I was doing some work for a 
company that processed credit cards. They had asked me to give 
them a talk about productivity improvement . in this credit card 
business. I said, "Well, tell me the different departments that 
you have." And they had one department called Research and 
Reconciliation. I said, "Well, what does that mean?" They 
said, "Well, that means that the statement got out with errors 
and now we are having somebody investigate the error and correct 
i t. " 

I said, "In factories we call that the Rework and Scrap 
Department." They said, "Oh, we can't use those words here. We 
are a professional outfit. This is a financial institution." I 
said, "The concept is the same." "No, it isn't." "Well," I 
said, "let's not argue about whether the concept is the same. 
You are asking me to help you determine how we can have greater 
output of this department. That's the wrong question. The 
question is, why is that department there in the first place? 
Why are you generating so many errors that you have to have 125 
people in the Research and Reconciliation Department?" 

If we are not careful, and just look at things from a 
pure productivity point of view, all we will be talking about is 
how to do the wrong things more efficiently, rather than how to 
prevent that kind of activity from occurring in the first t,lace. 

And by the way, that is going to be a recurring theme 
with many of us here, that prevention is one of the things we 
should be thinking about again and again and again. 
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I was talking to an attorney a couple of days ago who 
told me he worked for a company in which they were required to 
review contracts four times. The attorney had to review it, and 
the secretary had to review it, each four times. They were 
supposed to check on each other and to initial off at the end of 
each review. 

It seems to me that there is a lot of detection involved 
there. Sure, they were trying to prevent some disasters later. 
But the question that you'd really want to ask in a case like 
that is, is there any method that you can use to prevent that 
kind of activity? 

We also know from a motivational point of view that if 
you know that others are going to check what you are doing, that 
you are less likely to do your work with care. In some 
experiments where we have only one checker we find the outgoing 
quality is better than if we have two. The second depends on 
the first and the first on the second. Not always, but this is 
something for you to think about, to experiment, to see whether 
you have this kind of activity imbedded in the work that you are 
doing . 

Now, going back to the statement that everyone has two 
jobs, doing the job you are assigned and then improving on it. 
That is quite a new concept, because many people would like to 
say, "I would like to learn how to do my job perfectly, settle 
into a routine, and just stay at that. Nobody is going to 
hassle me. I have got other things I would like to think 
about." Instead, what we are saying to them is, change is a way 
of life. Every job can be improved. Every one of us has an 
internal customer whose requirements will change over time. The 
target we are trying to hit during our career is a moving 
target. This means that there must be ways to improve what we 
are doing over time. That is what we are interested in, to get 
everybody to look at it from that point of view. 

Jack Welch, who is chairman of General Electric, puts it 
this way: "If it ain't broke, fix it." Because if vou wait 
until it is broken, it is too late. So it is not busywork 
trying to improve on things that are supposedly okay. But we 
are saying that we can better satisfy others in our agencies or 
department j. We can better satisfy customers if we take the 
point of view of never-ending improvement. 

And, you see, that is contrary to what we might have 
said in the past. So one of the things that the quality 
assurance sciences bring to us is a new way of looking at our 
managerial and professional responsibilities, rather than the 
traditional way we might have been taught in the past. It is a 
new way of thinking. 

Now, there are some new tools involved, too, but we are 
talking about a new way of thinking about our jobs. 

Now, let me just get into some aspects of the quality 
sciences. One of the first things that we sometimes talk about 
is that in commercial organizations when we use the word 
"customer" we mean somebody ortside the firm. We do market 
research, and we try to find out how well we are satisfying our 
customers. 
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Many years ago Dr. ishikawa of Japan was talking to a 
group of people in a steel mill. The people in one department 
were complaining about the quality of the intermediate product 
they were getting from another. He talked to the people in the 
two departments or sections of the production line. He said, 
"You in the second department who receive the materials from the 
first should feed back information; tell them what your 
requirements are. You are the customer of that other 
department. The way you are running this plant now, you are 
trying to show a great versatility by making do with whatever 
you get. And likewise, you in the serving department are simply 
viewing them as being stuck with whatever quality of products or 
intermediate that you give them, because they have no option to 
get it fr.om anyone else. And we should work together as a team 
and develop normal customer-supplier relationships, meaning 
trust, respect, and demand for quality levels." 

And so he talked about the internal customer and the 
internal supplier, and trying to get that concept within all 
departments of an organization. Each of us is viewed as having 
certain customers and certain suppliers. People who feed you 
information or reports or decisions *x provide other things for 
you are your internal suppliers. And x-iewise, you provide 
requests, you provide assignments, you provide whatever it is 
you provide. And, so, look at organizations in terms of a 
series of customer-supplier interrelationships. Each of us here 
is a customer of someone else's work. And each person here also 
supplies direction, or requests, or reports, or whatever it is 
that you do. 

We would like everyone in the organization to develop 
that outlook, no matter where they are in the organization 
chart, including Secretary Bennett or any other Secretaries of 
the departments that are represented here. Sometimes they are 
surprised at how many people there are with whom they deal; how 
little they know about their requirements (their criteria, using 
Dr. Greer's terminology), and how well we let them know what we 
want in measurable terms. 

Let me give you an example. I was working with a bank 
in which one of the tellers did her job without error, did it 
quickly, but she wasn't a very friendly person. Customers 
complained about her. Her bosses went up and said, "You are 
doing a good job, but be friendly." She said, "I am trying to 
be friendly. My family thinks I am friendly; my neighbors think 
I am friendly. What does friendly mean?" 

So, if we establish criteria we have to have an 
operational definition. There has got to be some way that the 
person in the organization knows what we mean. Otherwise, 
communications and improvements can't occur. And some of the 
things that we have in mind that we would like other people to 
do are very difficult to describe. We have to work very 
carefully with each other until we can finally find out what it 
is that they have in mind. So, operational definitions are 
extremely important and sometimes difficult to come by. If we 
are not careful, we grab those that are easy to measure first, 
rather than those that are significant. 
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Let me give you an example. You have probably heard the 
story about the drunk who was wandering around at night trying 
to find his lost billfold. He was found searching underneath a 
lamppost. When he was asked why he was looking for his billfold 
there, he said, "Well, that's where the light is." It had 
nothing to do with where he lost his billfold. 

That's the way it is sometimes when we develop measures. 
We develop measures that are easy, but are not the significant 
measures for those things we are trying to really measure in an 
organization. And so that is another part of the challenge that 
we have, and it requires great communication with people with 
whom .we work. 

Now, another problem that we have when we look at this 
customer-supplier relationship is, what are the requirements of 
these customers, and do they vary over time, and how can we take 
this into account? Your customer might be the person who 
receives your report or receives information from you or 
assignments, whatever. And let's look at some aspects of it. 

Let's say that you and I decide to buy a car. We buy 
this car. We are pleased with it. So we say our requirements 
are met. But, during the next couple of years all sorts of 
things fall apart on this car. And now how do we view the 
quality of this car? Not so good. In some cases we might say, 
well, there are two different aspects. One is quality when you 
buy it and the other is quality during use. 

Most customers aren't that sophisticated. What they are 
simply talking about is, are my requirements met during the 
lifetime when I am using the service or during the period when I 
am using the service or product? In the case of education, am I 
happy when I graduate; am I happy five years from now; and, am I 
happy ten years from now? Do I feel that what has been given me 
satisfies my requirements, not just at graduation, but during 
some reasonable length of time afterwards? 

Or, if you are writing a policy and procedures manual, 
or some sort of a brochure that is to be sent out to the public 
at large or to various schools, and the question is, how 
perishable are the data? Is it useful now? Will it be useful a 
year from now, two years from now, or whenever? 

Some things, by their very nature, are designed only to 
have a short life. For others, we have to think about the 
normal use of the product or service chat we provide. We are 
expected to anticipate those requirements over time; not just 
initially, but over time. If you are developing reports and a 
management information system, are you anticipating what kind of 
reports will be used over the next five years, not simply what 
might be used in the months or weeks ahead? 

And so, quality has this dimension of not just initial 
satisfaction, but satisfaction over a certain period of time. 
Technically, professionals in the area of the quality sciences 
might separate those and give them different names, but I think 
most people aren't interested in talking about two kinds of 
quality. They are simply talking about some kind of 
satisfaction tr.at they get, and that is all they are interested 
in. Another very interesting comment about quality, or a way of 
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looking at the quality of anything, is to look at it from what 
is sometimes called a process point of view. That is, any work 
that has a series of steps can be viewed as a process. 

We are not talking about a manufacturing process. We 
are talking about any kind of process. As an example, a 
switchboard operator receives a phone call and an inquiry is 
made to try and get a given service. What are the steps that he 
or she goes through in order to transfer that call that came in 
to the right person, and then subsequently, when that person got 
the call, whether they have to transfer it to someone else or 
they in fact »//ere the right party? 

In an organization with whom I worked recently, 60 
percent of the calls had to be transferred to a second person. 
If you were to walk by the switchboard operator, would that have 
occurred to you? If you hadn't studied the system, would you 
have ever known what was going on? The switchboard operator was 
a very pleasant person. There were no complaints about what the 
switchboard operator was doing, but when people in the 
organization dug into what was going on, there was 60 percent 
rework for that process. 

These are the kind of things we would like to improve, 
even very simple, basic things in an organization. Sometimes 
when we analyze processes or look at professional or 
administrative or management work or technical work we do it in 
terms of a series of steps. We learn a lot about those 
processes that might escape us had we not done so. So, one of 
the things that we like to do in quality improvement is to have 
groups of people within organizations, people who are familiar 
with the work that goes on in those organizations, to do that 
sort of analysis, to do flow charting. They try to find out 
what is really going on in the process versus what people think 
is going on and what they would like going on. 

Sometimes we have an interesting exercise. We would 
say, describe the steps necessary to initiate a new policy or 
procedure that we have in an organization. Tell us how it 
works. We have everybody go through the flow charting. And 
then we would say, okay, now that each person has done this 
individually, now let's bring them all together and compare 
them. People say, "You are wrong, it doesn't work that way, it 
works this way." Pretty soon we find that there is disagreement 
within the organization, because we have not studied it 
carefully, we have not determined the optimum, we have not 
indicated how we handle exceptions, and two things will happen. 

In some cases, we have two groups who feel they have the 
primary responsibility, in which case we have politics. In 
other cases, we have nr. one who feels they have the primary 
responsibility for a certain part of that operation, in which 
case we have chaos; things slip through the cracks. Somehow we 
cover them afterwards, but we don't anticipate them. 

So, this is an important way of looking at 
organizations. If we want to go a bit further in this whole 
business of process analysis, one of the things we might do is 
to look at certain activities. We could keep track of 
complaints and then see if patterns occur. Let me give you an 
example. 
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We were working with an architectural and engineering 
firm in San Francisco who made drawings and specifications. 
These were sent to the Middle East. About six months later 
construction would start. There are a number of interesting 
things that occurred. 

One was, engineers who were working on the other side of 
the wall to the left of me thought, well, our layout can be 
improved if we just close off the door that is here. We really 
don't need that door because there are several other doors in 
this room. In the meantime, the engineers on each of the other 
walls was making the same decision, and now the blueprint got to 
the Middle East. Here is a room without doors, and they say, 
"How could this happen? What is going on in San Francisco?" 
And so they would start sending nasty letters back, and phone 
complaints one on one. Pretty soon it got to the point where 
the blueprints were actually viewed as a suggestion and not as a 
way to construct anything. Well, that was the ultimate chaos! 

Now, what could we have done, and what did we do? Every 
time they discovered something that went wrong they wrote it up 
on a little slip and put it in a box. And once a month they 
cleared this box out and they saw a pattern. Some of the 
problems occurred quite often, some less often, and so on. A 
pattern of priorities developed. Now they could go back to the 
people in San Francisco and say, these are the things that 
happen most often. It wasn't griping on a one by one basis. 
Patterns were seen, and changes could be made in training, or 
procedures, or the way they did things in San Francisco. 

And, by the way, if you wish, you could call that a 
"Pareto" distribution, depending on whether you like the Italian 
pronunciation or not. And this is simply a ranking of the 
problems in a priority fashion. Some of you might simply say, 
why give it a fancy name, you know, the top ten, whatever. 
Well, I don't really care what name you give to it. Just that 
you have some sense of priority. 

What does that have to do with a flow chart? If we find 
that there are certain problems that occur much more frequently, 
we then try to go back into the work process and find out where 
they started. And that is a start of prevention. You see, we 
could put an extra inspector on the end to check the drawings in 
the case I gave. But instead, we could go back and find out 
where they get their start, what is the cause of that problem; 
can we remove it once and forever? And, so in many cases we 
have two options. We have the prevention option or we have the 
detection option. 

Why do employees in an organization prefer the 
prevention option? Nobody likes to spend a day or a week or an 
hour doing things in the wrong way and then have somebody come 
back and say, sorry, you wasted your time. This whole day's 
effort, this whole week's efforts, or an hour's effort, is all 
wrong. You don't feel good about yourself. You don't feel good 
about the people who gave you the assignment, who didn't help 
you prevent the problem. Prevention is one of the concepts we 
are interested in. We are interested in the satisfaction that 
people have in doing work right the first time. That is more 
than a slogan! There are ways of helping people with the 
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methods of the quality sciences. One of the ways that people in 
the quality sciences go about it is through drawing these flow 
charts, trying to find out points in the flow chart where there 
is a potential for problems, or problems in fact have occurred, 
and then to find ways of removing the cause of those problems. 

Another thing that we would like to do is, when we look 
at the process, look at all the steps we have in the process, 
all the checks we have, to make sure that all the things that we 
do are related to some things that our customer eventually wants 
and we have no extraneous work. Let me give you an example. 

There is the story — perhaps it is apocryphal — of the 
guard standing next to a bench at the Arch of Triumph. Somebody 
who was looking at quality improvement asked, "Why do we have a 
guard there?" The French army, so the story goes, never throws 
away records. They went back into history and found out that 
about 100 years ago, or whenever that arch was put up, the bench 
was painted and a guard was put there to make sure nobody sat 
down on the bench. 

And now, 100 years later, all the relatives and friends 
and ancestors of that guard are still there. Tne question for 
you is, in your own organization, did you have a problem occur 
that required you to put an extra check on or an extra procedure 
in, but now you removed the cause of that problem, but the check 
or procedure is still there? Question: where is that French 
guard in your agency or department? How can we find that French 
guard and get rid of him, because he is adding to cost and not 
to value, and if we have limited resources we would rather use 
those resources in ways that will add to better quality of 
service. 

In staff service or service organizations, we sometimes 
have difficulty in developing quality measures. One of the 
measures that is quite obvious is a measure of timeliness. You 
want something done in a certain period of time. The question 
is, do those who work on it get it done in time? And if they 
do, you say, well, their quality of service is good. If they 
don't, you are unhappy. 

So, in many kinds of professional work, in many service 
organizations, even in many other kinds of organizations where 
you are handling materials, such as the Postal Service, 
timeliness is a very important measure. There are ways that we 
can go about improving timeliness. 

A second measure is a little more deceptive. And that 
is a measure that we sometimes call first time through. It is 
not error rate. One of the things you might talk about is how 
many errors are created in a given department, how much mail is 
missorted, gets to the wrong place. I am not talking about 
that. In many cases, we keep doing things ovei and over again, 
so many times that it always gets rigiit, but at great cost. And 
so we think our quality is good. What we are really talking 
about is how much makes it through the first time without being 
recycled? That is a first time through measure. 

In manufacturing, if 100 television sets come down the 
line, how many repairs or modifications do we have to make per 
100 sets before they get out? In law firms, in accounting 
firms, in mail departments, with people who are developing 
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policies and procedures, doing all sorts of administrative work, 
the question is, how many times does it cycle back before the 
report finally gets out? 

That is a first time through measure. It always gets 
out right, or maybe most of the time, but you see what we miss 
is the intention of getting it done right the first time. And 
so I like to use that as my second measure, a measure of first 
time through. How many cycles does it take to get any work 
through the first time, to get it out so there are no errors? 

Now, the last measure that I sometimes use, and there 
are a variety of forms. It depends on the application. I 
sometimes talk about a measure of relevance or satisfaction or 
style. It all depends on the application. Let me give you 
several examples. Let's say that you are my friendly mortgage 
officer, and I go to your bank, and I want to get a loan. You 
do it quickly. Great for timeliness. No errors; processed 
quickly. But while I am there, I notice the loan application 
forms of all of you in the front row of this conference hall on 
your desk. I can see how much debt you have, what your salary 
is, and all kinds of personal information about you. 

Now, what is the quality of that loan operation? Done 
quickly. No errors. Am I satisfied? No. Confidentiality has 
been breached. 

Let me give you another example. Let's say you called 
my office for some information. It is done quickly. No errors. 
But the secretary is very unfriendly. She is in a hurry to get 
to lunch, or in a hurry to leave for the day. Or is not having 
a very good day. It makes you feel that you are part of the 
problem for asking such a difficult question. But she gets it 
to you without error and quickly. How do you feel about doing 
business with me? Are we a quality house? You'd say, "no." 

So what I am saying is, there are many dimensions of 
quality when you are talking about administrative and 
professional work. It is not just done in a timely fashion, not 
just done without error. But there might be another measure of 
overall satisfaction, style, service, confidentiality, depending 
on the problem we have. And so one of the things that it is 
important to remember is that an overall measure, even though we 
would like it, is very difficult to come by, and economists have 
discovered this years ago. 

When the Consumer Price Index was first brought out in 
the 1920s, economists all over the land complained about all the 
shortcomings of the Consumer Price Index. Through the years we 
have learned that the Consumer Price Index, used properly with 
other indicators, is a very helpful indicator of how the economy 
is going. And so it is with quality measures. We might find 
that we would need several quality measures. I have given you a 
few that would give you an indication of how service in a given 
department or agency or whatever might be going. 

Another way that we sometimes look at the process is in 
terras of the degree of variability that occurs in that process. 
Perhaps we have new people coming in, so the results aren't 
predictable. Quality of the output varies quite a bit. So one 
of the things that we want to do is to look at things from the 
point of view of variabilities at various stages, not only 
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inputs to the process, but the variability that we might have at 
various stages in the output. That is the statistical approach 
that Dr. Greer talked about? using concepts of statistical 
process control. I don't want to talk about that any more other 
than tc simply say that this is an important concept that people 
in the quality assurance and quality sciences use. It is .aore 
than the deterministic concept I used earlier. We had different 
steps in a process and assumed everything moved from step to 
step without any trouble. Now, the other vision of the world 
that I introduced is, now that there might be quite a bit of 
variability of the work coming into a department, or process, 
variability in the work that goes at each stage for whatever 
reason. And now, how do we control that variability? 

There are a number of ways that we can do that. And for 
some of you here that might be an important kind of application. 
If you think you are interested in those things, there are many 
seminars on statistical process controls that you might attend. 
Some are offered by the American Society for Quality Control, or 
other groups, local universities, and so on. 

The next concept I would like to talk about is what we 
might call a systems audit. Now, let's talk about financial 
audits for a moment, and let's see how the financial part of the 
world operates. In commercial companies, the way the books are 
kept is based on a series of generally accepted accounting 
principles developed by the accounting profession. People who 
develop reports that are to be used by tax authorities, by 
bankers, by stockholders, by companies registered with the 
Securities and Exchange Commission, all must follow these 
generally accepted accounting principles. From time to time, 
some opinions are written by the Financial Accounting Standards 
Board that might indicate how these principles are to be applied 
in certain areas. Reports for the managers of the firm are not 
covered by any external body. The way the books are kept for 
running the business has nothing to do with the way the books 
are kept for these external to the firm. 

So, to reduce complexity, what most companies do, and 
most organizations do, is try to force that external system into 
the managerial system and hope that is works. They don't want 
to develop two separate sets of accounting systems; one for 
those external, one for those internal. 

External auditors are brought in to evaluate whether 
generally accepted accounting principles are followed if your 
firm is listed on a stock exchange. If you look at the annual 
report of a company there are only two things you have to worry 
about. If there are two paragraphs written by the auditors it 
means everything is okay. They say we have taken a sample of 
the transactions of what goes on with the company and we found 
that they have followed generally accepted accounting 
principles, so we give you a clean bill of health. 

But if there are more than two paragraphs, then they 
will tell you that they have some reservations, and then they 
start spelling out what those reservations are. So there is a 
set of standards as to what a sound accounting system should be. 
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Now, in the quality area, we have the same kinds of 
things. Remember in the beginning I said that there were two 
aspects of a job, one is doing the job right, and then the 
second is improvement. In many organizations we have a quality 
assurance system, a set of various policies and procedures, 
methods, practices, all listed, and that tells you how you are 
supposed to do the job. It doesn't tell you how you are to 
improve it. It just says that is the quality assurance system. 

From an auditing point of view there are several kinds 
of audits. One is the systems audit. Another is to look at the 
methods and procedures by which you measure or check work. And 
the thir^. is a practices audit, which is to make sure that 
people are doing what they are supposed to be doing. This is 
the kind of work that — the latter— that supervisors should be 
doing. Occasionally, perhaps, somebody internal to your agency 
might come in and ask people, what are you doing, can you show 
me where in one of the instruction manuals or policies and 
procedures it says you are supposed to be doing it in this way? 

Sometimes people drift, and they develop new procedures 
or guidelines that they don't document. That causes all kinds 
of trouble for organizations. 

You find people doing it one way here, another way 
there, or it is only in the mind of somebody. Somebody says it 
is okay to make that change but it is never documented, and so 
we have all kinds of confusion «ithin organizations. One of the 
things that we recommend is that internal to your organization 
you have a quality assurance function to do systems audits. 

Commercial manufacturing firms occasionally have an 
external consulting firm come in about every three years, not 
every year like they do in accounting, but about every three 
years to come in and do a systems audit of their quality system. 
To the best of my knowledge, governmental units have never done 
this. You rely on your internal audit groups within government 
to do this for yourself, whether within your agency or some 
overall body within government. The Government Accounting 
Office is close in its activities. 

A systems audit has a couple of parts to it. One is, it 
tells you whether or not you have a system that is generally 
agreed upon as a good system by professionals, not only in the 
U.S., but throughout the world. Professionals are in general 
agreement on what the elements of a sound quality system are. 
Frank Caplan, who you know very little about, who introduced the 
program this morning, has written a very fine book in this area. 
You might want to look at that some day. 

The first important point here is, is the system being 
followed? The second aspect of it is, do you have a cost 
effective way of doing it? And that is sometimes difficult for 
those within an organization to do by themselves, especially if 
they designed the system. They don't have the necessary 
objectivity, "hey might find it difficult to compare what is 
being done with what might optimally be done had somebody with 
greater experience or breadth of experience come in and looked 
at it. 



What's the objection to external audits or even internal 
quality audits? Well, you've got people checking on people who 
are checking on people who are checking, and it seems to be an 
endless kind of thing. We are not checking the work here, we 
are checking the system. We are evaluating the system. We 
usually are not talking about evaluating the work that is done 
by a given person. 

These are some of the other aspects that are helpful in 
assuring that the quality is what it should be, and Dr. Godfrey 
will be talking about some of these things to a much greater 
extent. 

When we look at problem solving in a professional 
environment, one of the first things we would like you to 
identify is who your customers are. Who are you internal 
customers? You might say, well, the people who cause trouble 
are my external customers. Well, let's forget them for a 
moment. Let's just take a narrower view and find out those 
within your particular agency or department who are your 
internal customers. They get whatever it is you produce, they 
get decisions, they get reports, get brochures, get funds, 
whatever it is that you do: even schedules and sometimes 
deadlines. 

We would like to put you at the center of a diagram, 
which we will call the sun diagram. Draw a circle, put yourself 
in the miudle. That is your job. Now we would like you to draw 
arrows out from that circle to all the different people who get 
information, reports, decisions, funds, or whatever it is that 
you give to them, and draw circles with their functions at the 
end of those arrows. You now have a sun diagram, and it tells 
an employee who their internal customers are and those to whom 
he/she supplies information, reports, whatever it is we supply. 
Those are the outflows. 

At the opposite end of each arrow we look at the inflows 
to us. First of all, we had the outflows from us. Now each of 
those people will be feeding something back to us. What might 
that be? Perhaps feedback on the report. Maybe a report. If I 
ask somebody to do something, I view them as a supplier. They 
view me as a customer for studies, for requests, whatever. And 
so they will evaluate how well I make these requests. Do I make 
them timely? How do I do it? 

And so now we see the interrelationship between 
ourselves and those to whom we supply information or make 
requests. They, in turn, give a feedback to us. This is very 
.useful in seeing the relationships that we have within an 
organization. Of course, the next question we would ask is, 
well, what are all those outflows and what are all the inflows? 
And how well are those done in terms of timeliness, first time 
through, satisfaction? We can make quality measures and we can 
diagnose organizations in terms of how well this is being done. 

Another tool that we sometimes use is called a cause and 
effect diagram. Now, you know that old story — or perhaps you 
don't— that was told by Dr. Shewhart many years ago, one of the 
great founders of the field of qualiy, about the fellow who 
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drank scotch and water, got drunk, bourbon and water, got drunk, 
rye and water, got drunk, and stopped drinking the common 
ingredient, water. Now, of course, what is wrong with that? 
The solution had nothing to do with the cause I But there are 
many problems like that in government policy. Kids aren't doing 
well in school. Build public housing. Build public housing; 
kids still aren't doing well in school. 

Well, in that case one of the problems is that there 
might be a multiplicity of causes, and in some cases we try to 
oversimplify problems and pretend there is just one cause and 
one effect. And so one of the things that people in the quality 
sciences try to do is sort out problems in terms of those where 
there is a single or dominant cause versus those in which there 
are a variety of causes. In some cases the causes even interact 
with each other, and have a multiplier effect. 

We know that effect in medicine, that by itself we can 
say that asprin is quite good, but asprin for certain kids with 
certain illnesses at certain age groups can produce deleterious 
effects. An interaction has occurred. A particular malady, a 
particular age, and a substance that is ordinarily good reacts 
in an unforeseen way, and that is what we have to concern 
ourselves with. 

Okay, now, what about standards in education? Dr. Greer 
talked about criteria. These are extremely important whenever 
you talk about quality. Standards in terms of tests. Standards 
in terms of what it is you are trying to deliver. Standards in 
terms of how you try to administer something. Standards for 
school boards. Standards for business managers of school 
systems. Standards at all levels. 

Now, for standards to be effective, they must be 
mutually agreed upon. They shouldn't be coercive. There should 
be a quick feedback, provided in a way that is not threatening. 
They should be given in a way that signals that we are trying to 
help improve the system rather than put someone down. Again, 
Dr. Greer implied this a bit in terms of how some of the 
evaluations of some of these criteria work. In many cases 
people aren't interested in standards, or even interested in 
small group processes and quality improvement, if they feel that 
the output will be unfair, even though the process used is fair, 
so there are two things involved. 

One is, is the process by which you make the evaluation 
fair, and then, is the output fair? I, as an individual, might 
not want to get involved in establishing criteria or involved in 
small group activities and improving services, even though I 
might feel that we use democratic processes. If 80 percent 
voted in favor of some program, I might find that that 
democratic process isn't good enough. Majority rule, or 80 
percent, or whatever, might result in something that is quite 
damaging to me, so I don't want to have any part of it. So 
think about those two parts whenever you are working with 
criteria and you are working with evaluation. 

Even though the process is fair, someone might not Ike 
what you are going to do with the output. One of the reasons 
that we have small group activities, such as quality circles, 




27 



fail to a great extent in industry is because many employees are 
worried about whether those two aspects are being handled in a 
fair way. 

What is the potential of the quality sciences in running 
government and in the educational curriculum? I have been 
concentrating on a number of suggestions for how these concepts 
might be used in running your agencies and departments. Let me 
give you some curriculum suggestions. 

The first is, when we talked about customers and 
satisfying customers, at the kindergarten level or pre-school 
level we can get children to think about what they like, and to 
be aware of what their requirements are. And then, next, to get 
them to be aware of how can people sell what they like? In 
other words, the marketplace. How do they know what it is that 
you want? You know what you want, but how do they know it? How 
does that information get to them? And then as they have 
part-time jobs in the neighborhood, one of the things we would 
like to get them to understand is this customer-supplier 
relationship. How is satisfaction developed in business, 
government, and in the media? We do that in the high schools. 
Parts are handled earlier. 

Then, the next concept we would like to get them 
concerned with is teamwork. We are noted in this nation of 
having a heroes concept and not being much on teamwork. We 
should be aware at the lowest levels of what to do with others, 
what kind of things would we like to do with others in 
pre-school and kindergarten. In the first grade, what can. you 
do to make it easier for others in your classroom when you are 
involved in play or group activity? At the sixth grade, how can 
you get your ideas listened to by others? And even teaching 
concepts of brainstorming and the nominal group process. And 
then in high school, how to go about complex group problem 
solving and project work. 

We then like to get concepts of measurement introduced, 
in pre-school and kindergarten, more or less, in the fourth 
grade, some concepts of graphing, perhaps temperature, cause and 
effect, puzzles. In the sixth grade, group problem solving, 
prioritizing, and the concept of standards. We then like to 
introduce concepts of variability. And in pre-school and 
kindergarten look at height and time, have kids line up and show 
them that some are taller than others. The variation of 
heights. More time, less time. In the fourth grade, concept of 
seasons and to develop various models or ways to describe 
things, bring in concepts of models. In the eighth grade, 
averages, graphing, ranges. The concept that problems have more 
than one cause. The tenth grade, the concept of interactions 
between causes. 

And then systems concepts, the fifth category, in 
pre-school and kindergarten, the order in which you put on your 
clothes; the order in which you put on your shoes and then tie 
them. You don't tie them before you put them on. in other 
words, the concept that there is a right sequence to do things. 
And then, as we get in the fourth grade, things the family has 
to do to shop, clean, or get people places, looking at the steps 
in the process, parallel processes that occur in family life. 
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In the eighth grade, staggered processes, feedback, 
collective action, some concepts of computers. 

Well, I hope I have given you some ideas on how some of 
these concepts of the quality assurance sciences can be used, 
not only in the management of your own work in your own 
departments, but some that might be used in education at various 
steps along the way. 

Thank you very much, ladies and gentlemen. 



Mr. Walser: 

We appreciate very much your presentation. I think it 
is now time in the process he was talking about for a little 
customer feedback. I have a question or two here. I have one 
that has been delivered to us. There are also microphones in 
the aisles, as you can see, and those are there for your 
convenience if you wish to engage in a discussion, a little 
dialogue, question and answer. 

This seminar is being recorded so that we can produce 
some reports from it, or extract information as would be 
appropriate from these meetings. So it is now your time to ask 
Mr. Golomski a question or give him some of your reactions. And 
here is the first one. 

Question: One, given the absence of the bottom line in 
government, and two, given the requirement of top management 
support to implement and institutionalize new concepts, and 
three, given that top government managers know and care little 
about improving productivity (efficiency, quality, and 
timeliness), what makes you believe that quality sciences can be 
effectively applied to ■ government programs? 

Mr. Golomski: 

That is a superb question. I have run into it as I have 
worked with various governmental groups, and currently am 
working with the city of Madison, Wisconsin, and the city of 
Chicago in some aspects of government, anr* as you know, Chicago 
government is as interesting as any theat : on Broadway, so it 
involves the participation of a lot of d:; *rse segments of the 
community. I approach it on three different levels. 

The first is an individual level. I find very few 
people, and I mean very, very few who really don't care whether 
they do worthwhile work during the day or not. There are very, 
ve::y few people who would enjoy working all day long and finding 
out that what they had done wasn't worthwhile. There is a 
tremendous amount of pride in people at all levels. I don't 
care what their family background is, what their educational 
background is. People like to do work well. I have even found 
this in working with people in prison, who might not have any 
hope of ever getting out. They still want to do work well. 
They don't like the idea that they would spend the whole day or 
several hours or several minutes doing things that weren't 
worthwhile'. 
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So, the first level is the individual level. The second 
level is the intermediate manager level. And again, people at 
this level find that they are constrained for budgets, that 
their upward career advancement is based on how well those who 
report to them do, and if they concentrate on simply 
ingratiating themselves to people at levels above, they will 
never make it. They have to have the support of those who work 
for them, and so the more they can get them involved in quality 
improvement, the better off they will be. One of the most 
marvelous reputations that anyone can have is to be a great 
developer of people, even to the extent that occasionally 
somebody will pass you up and move onward in the organization. 
That is still a marvelous reputation to have, because when they 
have moved up they will recognize what a good job you do in 
developing people, so that is the second level. 

Thirdly, at the uppermost levels in the government, I 
find that there is more of an interest in productivity in most 
cases than quality. But in recent years economists have come to 
realize that through quality improvements you can get 
productivity improvements. Furthermore, most people are more 
interested in participating in a quality improvement process 
than productivity improvement. 

And so we find that, even though very few people at 
uppermost levels are talking about productivity, when we get to 
talk to them we convert them to also think about quality. They 
will eventually find the quality improvement program within the 
organization will result in productivity improvement anyway. 
Now, in terms of a lot of additional dollars, there are not a 
lot of additional dollars. There usually are a few front end 
dollars, but after that programs pay for themselves. I can tell 
you tbis in working with all sorts of government organizations. 
There are the initial front end training dollars, but after that 
people continue the process by themselves. Have I answered the 
question, or do you think I am trying to avoid it or throw out a 
lot of platitudes? Yes? 

Dr. Andrew Adams: 

I am a little confused by your answer. You have stated 
that some are interested in productivity and may or may not be 
interested in quality. I look at these two as the same, not one 
or the other. In other words, productivity to me is the 
definition of organizational goals that you want to apply, and 
if those goals have quality built in, then productivity would be 
quality automatically; non-productivity would be not making the 
quality part of our goal. 

Mr. Golomski: 

Yes, the question is, there seemed to be some confusion 
in my reply, that quality and productivity are associated with 
each other and I didn't imply it. They certainly are associated 
with each other. But what I am saying is that when there are 
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public pronouncements from people at the upper parts of 
organization, often times they are stated only in productivity 
language, rather than in the language of quality. 

And so one of the things we are trying to do is to get 
people at upper levels of the organization to understand that 
quality improvement will lead to productivity improvement. We 
haven't accomplished in all parts of the industry or government, 
not only in the U.S. but in other countries of the world. And 
that is one of the things we hope that people understand. 

There are a few cases where a person might have a 
greater interest in productivity, perhaps in a given piece of 
equipment that scores tests or something, and the primary 
interest is productivi ty, but even there you might get some 
quality advantages as well. Sometimes you might not. 

I was in a hotel recently in which there was one of 
these forms that is to be scored automatically. There are all 
sorts of nasty statements on the form, such as, "be careful 
about- the way you fill in these ovals because the machine can't 
score them correctly." Statements like that that made you feel 
like, why do I want to fill this out at all? All they are 
concerned with is productivity of scoring, not my input. 

So, some of these things can be viewed from that point 
of view, but no, I agree with you 100 percent that, yes, quality 
improvement leads to productivity improvement. 

Yes, sir. 

Mr. Ullman: 

I'm Neil Ullman. I would like to make an observation, 
kind of comment, and then I have a question. The observation 
was, your comments about people who don't want to work or who 
don't really care, and unfortunately there is that conflict with 
what Dr. Greer was talking about with his child, and 
unfortunately, that syndrome seems to be prevalent with a lot of 
young people. At least, I see that in an observational way. I 
think there are some serious problems, and I don't know how we 
tackle that issue. 

But the question I really would like to touch on is, 
when you mentioned about the audit, and in the public school 
sector, particularly on the high school nd the college level, 
you have the accreditation process, ana then you have the 
accreditation process in specific areas. How do you view the 
audit process that you are talking about in its relation to the 
accreditation process, and can the two be merged in some sort of 
fashion so that it might be more productive in accomplishing 
multi-tasks on that? 

Mr. Golomski: 

I will answer the second question on the accreditation 
process, first. I have a little familiarity with schools of 
business. Sortie of the best schools of business choose not to be 
accredited because *;he standards are too inflexible. And so one 
of the strange things you have is that the worst schools in the 
process aren't accredited, nor are the best schools. When you 



have an accreditation process that just meets at the middle 
level, then you can stamp out innovation, so that is one of the 
concerns. Accreditation is good, but make sure that you have 
some opportunities for innovation. 

But the other part that accreditation doesn't get at 
that you would get in the quality audit, and that is the degree 
to which a place is administered well, and in terms of the 
quality systems that are in place for administering an 
institution of higher education. In my work as part of teams 
that have gone in to accredit colleges in certain areas, one of 
the things we never get at is some of these administrative 
aspects. We concentrate a great deal on curriculum, talk to a 
few students, a few faculty members, in terms of the degree to 
which prevention concepts are in place, I don't recall that ever 
even occurring to us. 

Mr. Ullman: 

Well, I think you mean in the regional accreditation, 
like in the middle states area. That, accrediting an 
institution as opposed to the curricula accreditation, it seems 
to me that there is a vehicle for modifying that because it does 
look at the financial aspects and the organization aspects, and 
it touches all different levels. 

Mr. Golomski: 

That is a good point, and there is an opportunity to 
improve that accreditation process. You are right on target, 
i'es? 

Mr. Scruggs: 

I'm Max Scruggs. It seems to me based on my experience 
in federal government, working in a couple of agencies over the 
past 20 years, that people are hired to do specific jobs. For 
example, those jobs may be in the statistical area, the legal 
area, the education program specialist area. Supervisors, 
middle managers, and top management get into their positions not 
by dint of their expertise in management, but because they are 
available or for political purposes. Our managers are not 
managers. They have no training, no real training, no 
substantial training in management, no experience to a great 
extent in management. Most of our branch chiefs, supervisors, 
and division directors have only supervisory training. What 
they care about is doing the work, the statistical work, or the 
legal work. They don't have sny interest or very little 
interest in improving productivity, quality, or timing. 

Mr. Golomski: 

That's a good point, and I am glad you brought it up 
with respect to my response, because one of the things that I 
find is that when we put people into these positions we can take 
two points of view. One is that your job as the person in 
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charge is to carry out the transactions of that organization, 
and what we are saying is, from a quality sciences point of 
view, that we would like people to view their occupations or 
professions as having another dimension. That is, transforming 
the work in the sense of Jack Welch's comment, "if it ain't 
broke, fix it." 

And so one of the things we would like to do is to have 
everybody look at their responsibilities from two points of 
view, not just managing, but leading, transforming an 
organization so it gets better. I find that there is a growing 
interest in government right now to do that. Whether we are 
doing it fast enough or in all parts is another thing. I have 
worked with large manufacturing organizations which have done a 
superb job. Then I talk to some supplier who tells me, "Look, 
that's just a lot of smoke. The purchasing agent I deal with is 
only interested in cost. He doesn't believe in quality." 

It is too bad. The message hasn't gotten down to that 
person or that person's supervisor. And so, to get large, 
complex organizations to work ideally in all areas is very 
difficult. When we start the quality improvement process, in 
most cases, we like to start at the top and then move level by 
level down through the organization, not to move too fast, until 
the level above has bought in- That is difficult in many 
governmental units because people ire not in the job too long. 

Nonetheless, we do see hope in these areas, and many 
organizations are trying to do this. In working with the city 
of Madison, Wisconsin, the mayor turns out to be a very 
articulate spokesman for quality improvement. He w?s here in 
Washington a few weeks ago talking to a national conference of 
mayors. The police chief was also talking to a national council 
of police chiefs. That police chief has a quality improvement 
program in the police department and he is having success. 

There are people among the aldermen, I am sure, who 
might think that this is just some sort of fad, but the people 
who are running those departments find it is very useful. People 
within them like the new management style better than they like 
the old, and so there is a real transformation. 

Police departments are among the most difficult because 
the more contact you have with them the worse your idea of how 
they operate. If you have never been in contact with the police 
department, you think they are great. If you get stopped for a 
traffic violation in the morning, you are a little unhappy with 
yourself. If you are stopped again at lunch, you think a little 
about them and yourself. If you are stopped again in the 
evening you think, "don't they have something better to do with 
their time?" 

So, you see, each of these governmental units has a 
different set of circumstances to deal with. Police departments 
have customers, who are the citizens at large and those who are 
arrested. We have two different segments, each to be handled in 
a different way. And so I think that's the way it is here. But 
it is possible. I can tell you, organizations are interested 
throughout the land and are doing a good job. Whether they are 



moving fast enough in Department A or B or wherever, eventually 
we get to the point where people understand, and Congressmen 
understand. 

Recently several of us were in a meeting at which 
Congressman Ritter spoke. He is an articulate spokesman for 
this concept. Now, how does he get this to all other folks? I 
don't know. But there is an interest. 

Mr. Walser: 

Thank you, Bill. Thank you very much. You will notice 
in your agenda that this afternoon, after all of the presenters 
have presented, there is a question and answer panel, where all 
of the presenters will be here, and you can engage in additional 
discussions and dialogues for clarification or observation. 

Right now it is time for a break. We would like you to 
be back at 11:30 so we can be prompt in starting the next 
session. I would encourage you to meet each other. I know 
there are many people from many different parts of the country 
as well as different organizations, and take advantage of it and 
introduce yourself to your neighbors. 
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Mr. Caplan: 



Our next speaker in his current incarnation is the 
chairman of Juran Institute, a prestigious educational 
organization founded by Dr. Juran, who has been very active in 
the quality sciences now for 50 years, I guess. Dr. Godfrey 
recently achieved that position. Vi i are delighted to have him 
with us today talking about quality of service and staff in a 
training environment. Blan. 



Presentation: Quality of Service and 
Staff in a Training Environment 

Dr. Godfrey: 

INTRODUCTION 

I am going to talk about three things today. I want to 
start with an explanation of why we, those who work in industry, 
are so interested in the subject of quality and productivity. 
Then I am going to cover in some detail what we in industry, 
especially in the education side of industry, are doing. I am 
going to try to relate this to what I think others in the 
education business can do. And I will give examples that will 
be, hopefully, very relevant to some of the work that is going 
on in federal government, state government, high schools, and 
even elementary schools. 

The reason that industry is so very concerned in quality 
and productivity is really a national interest. Let me start 
with a quote from the New York Times from February 7th, 1988: 
"America can't compete. That perception has slowly and 
painfully crept up on us in recent years. Many Americans hoped 
that the falling dollar would solve our problems by making 
American goods cheaper and therefore more salable abroad. 3ut 
recent trade figures make it clear that the cheaper dollar is by 
no means a panacea." 

This article in the New York Times by Daniel Sharp was a 
summary of the report of the 74th American Assembly, held 
November 19th through 22nd, 1987. The America Assembly was 
started by Dwight D. Eisenhower while president of Colombia 
University. He started the Assembly to address a major issue 
every, year. One year, for example, it was apartheid. One year 
it was nuclear disarmament; last year the subject was global 
competitiveness. The title of the report from the American 
Assembly is, "Running Out of Time: Reversing America's 
Declining Competitiveness." For three days during this 
conference, 65 leaders from business, labor, the media, 
academia, and government discussed some of the major issues 
concerning the ability of the United States to compete in the 
global marketplace. The first speaker was Richard Lamm, former 
Governor of Colorado. Let me quote from the American Assembly 
report. 
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"Once leaders in the world, American companies 
have lost command of markets to international 
competitors. Though macro-economic factors like 
exchange rate and trade policies have harmed our 
ability to compete, a strong case was made that 
these problems are chiefly the result of 
ineffective management practices as well as the 
cause of other problems. 

"There are businesses and markets in which U.S. 
companies no longer compete at all. Those who 
try to compete find that working harder is not 
enough, that fundamental changes are 
necessary. " 

Let me talk just a minute about what is the problem. As 
everybody in this room knows, we had record trade deficits, $170 
billion in 1987. We get excited when it is down to only $10 
billion a month, as it was last month. In product after 
product, we no longer compete; we don't even try. In many other 
products we have lost much of our former world market share, 
even industries we created are now dominated or being taken 
over. 

In the U.S. today there is no longer much debate on the 
problem, but there is still strong debate on the cause, and very 
much debate on the solution. There are many possible solutions. 
In the United States today we are trying most of them. 

The first is, we can devalue the dollar. We can, in 
effect, make each American worker work for half of what he or 
she was working for a few years ago. Cheap American labor 
should make our products and services more attractive to 
foreigners. But according to the International Herald Tribune 
you can now buy two Buicks in Belgium for the price of a single 
BMW, but the Belgians still prefer the BMW. 

With the recent devaluations of the dollar, American 
workers now earn only 83 percent of the average Japanese wages, 
and 6 percent less than West Germans. Workers in seven European 
countries now earn more than U.S. workers. Even before the 
recent dollar devaluations, companies like Sony were coming into 
the United States because of low-priced labor. In the Sony 
California television plant, American workers were earning 15 
percent below their Japanese counterparts and winning Sony's 
awards for the best quality televisions made in the world. 

Another solution is to sell some of the wealth we have 
stacked up in more prosperous times. We are a very wealthy 
nation. We have enormous resources in land, personal property, 
and raw materials, even junk. Two of our most profitable 
exports— -I think they now rank Number Two and Three— are scrap 
metal and recycled paper. We are selling art works. One Van 
Gogh recently brought almost $40 million. We are selling our 
land. Forty-seven percent of downtown Los Angeles is now owned 
by foreigners. Buildings and land in Manhatten are bringing 
record prices paid for by people in other countries with strong 
currencies and huge trade surpluses. 
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Time magazine had an interesting article the week of 
April 3rd. In the last two years the Japanese have purchased $3 
billion worth of Hawaii, more than the total foreign investment 
in Hawaii in 20 years. 

Another solution is to sell parts of our future. We can 
sell American-owned companies and business. We are doing this 
in record numbers. For the first time in history foreigners are 
investing more in America than Americans are investing abroad. 
Every day we read that a familiar American brand name is being 
sold off to raise cash. Bloomingdale' s is now a Canadian 
company. RCA Consumer Electronics is now French. Quasar and 
Firestone are Japanese. Westinghouse light bulbs are now Dutch. 
Many major appliances are now Swedish. 

Investment houses, banks, and service industries are 
being gobbled up by bargain hunters from all over the world, in 
the first four months of 1988, British companies alone bought 
124 American companies. 

Another solution is to borrow. We can borrow money from 
all over the world to pay for our trade imbalance and our huge 
government deficit. We are now the largest debtor nation in the 
world. Only a few short years ago we were the largest creditor 
in nation in the world. Japan is now the largest creditor nation 
in the world, with many of its credits in U.S. securities and 
bonds. The Wall Street Journal , on February 29th, 1988, had a 
very strong opinion. The title of the article was "Reagan's 
Legacy: America for Sale." 

"Steady rise of foreign investment in U.S. 
businesses and real estate may turn out to be 
the most controversial legacy of Reagonomics. 
Already the selling of America has become a 
hot topic among politicians and the press and 
the sale has barely begun. The economic 
policies of the last seven years ensure that 
the foreign appetite for American investment 
will continue to grow in the coming decade. 
The end result will be a dramatic reversal of 
the U.S. role in the world economy. 



"For more than 30 years after World War Two 
U.S. multinationals spread their capital 
around the globe. Now the nation finds itself 
on the receiving end. The American challenge 
to the world as Jean Jacques Servan-Schreiber 
described it two decades ago has become the 
world's challenge to America. Even the most 
optimistic economic analysts see U.S. trade 
deficits continuing for at least the next 
decade. Data Resources of Lexington, 
Massachusetts, projects trade deficits to the 
end of the century. 

"That means foreigners will continue to have 
surplus dollars to invest in America. The 
trend startles many Americans who grew up in a 
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nation that dominated global investment flows. 
Throughout the 1950s and 1960s giant U.S. 
multinationals invested S4 or $5 abroad for 
every $1 that foreigners invested here." 

There is another solution. We can revamp our industrial 
base. We can once again become the world's leading developer 
and manufacturer of products. We can once again produce more 
than we consume. We can once again export the highest quality 
products in the world and the lowest cost products in the world. 

This last option I would like to explore. I would like 
to talk about what companies are doing. The other options are 
already being tried by our friends here in Washington, by the 
most poorly managed companies in America, and by many Americans 
who have given up, by Americans who feel that we no longer can 
compete. 

So, what can be done? I would like to quote again from 
"Running Out of Time," the American Assembly report, about 
fundamental changes the American Assembly thinks are necessary. 

"An increasing number of American firms are now 
bringing back home the lessons of industrial 
masters such as W. Edwards Deming and Joseph M. 
Juran, who taught Japanese industry long ago 
when U.S. companies would not listen. Those 
lessons are deceptively simple. After hearing 
from experts for three days, we have collected 
some basic principles of what makes a firm 
competitive, the first of which is quality, an 
emphasis on making the product or service right 
the first time. 

"Together these principles are: Quality. This 
does not mean quality merely to specifications, 
but quality that improves constantly, quality 
that is characterized by constant innovations 
that create a loyal customer. It means 
achieving this attitude from top to bottom, from 
the boardroom to the factory floor. 

"Low Cost. This is not instead of quality but 
the result of quality. It may seem cheaper to 
shove as many products or services out as fast 
as possible, but if quality is ignored the cost 
in rework, scrap, supervision, and most of all, 
disappointed customers will be more expensive 
than any business can bear. 

"The Customer. It has to be customer-driven. 
The customer is part of the process. The 
business exists not merely to satisfy the 
customer's need today, but to anticipate their 
needs of tomorrow. 
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"Employee involvement. The successful business 
no longer sees employees as a cost of 
production, but as a resource for production. 
Although job uncertainty will never be 
eliminated, it must be recognized that long-term 
commitment of and to workers is at least, if not 
more, important that machinery or technology. 
Employee involvement in efforts to improve 
productivity and quality is vital, and they must 
also be able to share in the gains. 

"Continuous Improvement. This means never being 
satisfied, not only with the products and 
services, but with the way the organization is 
required in all of these activities. It means 
changing our attitude from America's traditional 
•if it ain't broke, don't fix it' to 'if it 
ain't perfect, don't leave it.' There are a 
number of American firms taking leadership and 
implementing these innovations, and there is a 
critical need to accelerate this process." 

Company after company in the United States, and those 
companies wishing, desperately wanting, to survive are reacting 
the same way. They are creating massive training programs to 
reeducate their engineers, their work force, their managers. 
They are training these people for the jobs they must do. IBM 
has created a "university" for corporate technical education 
with three major parts: Systems and Software Education, 
Engineering and Manufacturing Education, and Quality. 

Bell Laboratories and AT&T have created an entire 
curriculum: project management for all managers, quality and 
productivity management, quality planning, reliability 
prediction, reliability estimation, experimental design, robust 
product and process design, and quality and productivity 
improvement. They have an even more extensive software 
engineering program. 

No one in industry today doubts that education is 
absolutely the essential key for our survival. What I am going 
to give you in the rest of the talk is some examples of what 
companies have done and what we think is working. We are 
applying the same quality principles that we are using for our 
products and processes to our education processes. 

Just a little background on my own education experience. 
I started teaching as a government employee, a member of the 
United States Army. In the Federal Republic of Germany on my 
first assignment I taught courses ranging from everything from 
driver's ed to nuclear weapons assembly. Later, I headed a 
special task force working out of the Pentagon teaching 
inventory control in the Republic of Vietnam to American 
soldiers, Thai soldiers, and Korean soldiers. As you know, we 
were pumping billions and billions of dollars worth of materials 
into Vietnam. Most of these supplies simply disappeared. We 
had no idea where they were in the inventory or how to get them 
out. 
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Then, when I went to graduate school, I couldn't survive 
just on the GI Bill, so I became a teaching assistant and taught 
business statistics to sophomores and juniors in the business 
school as I got my Ph.D. in statistics. Later, I went to Bell 
Labs and thought my teaching career was over. But after many 
years of working in research on quality theory and technology, 
working on network performance and other things, we were asked 
by a top executive what the status of training in quality 
control and reliability was for our engineers. We had to answer 
honestly, we didn't know, but we'd find out. 

We did a survey, 1,000 questionnaires to Bell Labs 
management and engineers; 746 people returned their surveys. 
Over 70 percent said that a knowledge of quality control and 
reliability was critical to the success of their job, and over 
70 percent of those people said they were very poorly trained to 
do it. These courses were not covered in the universities, and 
in these areas they had very little idea of what they were 

doing. , 
We couldn't go back with that answer. The answer we nad 
to go back with was what we were going to do about it, and we 
started our first course. What has happened in the five years 
since is the list of courses mentioned previously. Bell Labs is 
trying to reeducate engineers hired from the best schools and 
universities—the number one school Bell Labs hires from is MIT; 
the second is Stanford, and so forth. These are the people 
taking these courses. They could not compete in their knowledge 
of quality control or engineering for design and manufacturing 
with the people from across the oceans, especially in Japan. 

HISTORY OF QUALITY 

I want to spend just a few minutes on my view of the 
history of quality. The exa ole I like best from inspection 
comes from Czarist Russia and the fabulous Faberge eggs. (See 
Exhibit A) Most of us think of Faberge as an individual 
creating the beautiful jewels that ended up in museums around 
the world. There were actually 200 people working for Faberge, 
apprentices and masters and so forth, and they'd present a jewel 
to Faberge, he would inspect it very, very carefully with a 
magnifying glass, and if it met his demanding standards, he 
would put his name on the bottom and present it to the Czar or 
Czarina. If it didn't meet his exacting standards, he smashed 
it with a hammer and gave the jewels back to the craftsperson to 
try again. This is the ultimate quality by inspection, and I 
think drives home the point on why quality by inspection, 
although effective— at least he didn't get his head cut off for 
producing bad quality for the Czar— is efficient. It is sort, 

scrap, and rework. 

Back in the 1920s we like to think that the Americans 
invented quality control, Walter Shewhart at Bell Labs even 
coined the terms quality control and quality assurance. There 
are many earlier examples. The one we use as the logo of our 
company points out the elements of quality control very simply. 
(See Exhibit B) 
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Our logo shows a worker on an Egyptian pyramid cutting a 
block. Another worker is measuring and giving instant feedback 
to the worker. The worker can adjust as he cuts the block, not 
after he has made it and has to throw it away or to cut it to a 
smaller block to use somewhere else. The instant feedback 
enables him to control the quality of his work as he does it. 
It is a basic principle of quality management. 

When we think about quality improvement, and continuous 
quality improvement, we think back to a chart that Dr. Juran 
made in 1976 where he shocked many people in the western world. 
He plotted the rate of improvement in japan and the rate of 
improvement in the United States, and he predicted that Japan 
was on its way to quality leadership. (See Exhibit C) The 
reason why so many people were shocked in 1976 was that most 
people still remembered "Made in Japan" meant junk. They 
remembered that toys given to you on Christmas Day that said 
"Made in Japan" didn't make it though Christmas Day. They 
remembered the stories even told by Japanese. Japanese 
businessmen used to say that the first thing they bought when 
they came to the United States was a ball point pen because ball 
point pens were too difficult for the Japanese to make. They 
only made two types, those that didn't write and those that 
leaked. They said you could always spot Japanese executives by 
the black badge on their shirt pockets. 

What Dr. Juran was noticing was that the Japanese were 
improving at a rate far faster than anything he had seen. If 
the Japanese were improving faster than the West, sooner or 
later the Japanese would pass the West, no matter how far behind 

they had started. 

When we look at this chart today, we can say it doesn t 
really matter whether it is West or East or Company A versus 
Company B. (See Exhibit D) If one company has a quality lead, 
say Company A, and is improving at a steady rate, but the other 
company, Company B, is improving at a very fast rate, sooner or 
later in time Company B will pass Company A. And if Company A 
doesn't react, Company B will widen the gap over time. In most 
industrial products it is about two years before the customers 
really understand who the leader is, and then there is a massive 
change in the buying patterns. Then, not only does Company A 
catch up, Company A must regain the lead and get the customers 
back on its side. Nowhere is that more clearly evident than the 
U.S. automotive industry, where the auto industry has bounced 
back. They are doing many things. As one professor from the 
University of Michigan said recently, and I think it's a very 
accurate and apt quote, "America's auto industry is now making 
some of its best cars equal to the worst Japanese car, and that 
is a major accomplishment." That is a very accurate statement. 
That is a major accomplishment. 

So, when we started examining what the Japanese did to 
improve quality we found several things. Upper management took 
a very strong leadership role. They provided training for every 
member of the company far beyond what we have ever thought 
about. They created a revolutionary rate of improvement. 
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So, what I'd like to look at is what are these breaks 
with tradition. First, let's look at hands-on leadership. Not 
upper management in charge delegating, but doing things. 
Massive training for the work force. Annual improvement goals 
for quality. The rest of today will focus on this training. 
Let me give you one example that knocked my socks off. This was 
a cover story in the New York Times in 1977. The Japanese have a 
Statistics Day every year. It is a national day, and they have 
a contest every year. In 1977 the contest was for the best new 
statistical graphic. They had 29,940 entries for the best way 
to display something graphically. The winners were five 
seven-year-old Japanese students, five young g<rls who had done 
a study on what makes children happy. They t interviewed 
their classmates, and they had asked questions such as, would 
you rate yourself very happy, average happy, or not so happy. 
Happiness was the dependent variable. Then they studied many 
other variables such as how much time mother spent with each 
child, whether mother read to each child each day, how much 
contact the child had with mother and father. 

They identified the key variable, which turned out to be 
the time mother spent with each child. This was further 
dependent on the numbers of brothers and sisters, the number of 
times read to, and other related variables. 

Then they created three pictures of Japanese dolls. One 
happy, one not so happy, and one sad. They printed numerical 
values on these dolls, so in this beautiful graphic of three 
pretty Japanese dolls you could not only see which one was happy 
but exactly why. The results of a study planned, designed, 
conducted, and analyzed by five seven-year-olds were clearly 
displayed in these innovative graphs. 

When we look at the key processes of quality of 
education, they are very similar to other key quality processes. 
You have to know who your customers are, what they want, and 
what they need. You have to develop and deliver this education, 
this training. You have to" measure the quality. You have to 
provide self-control so people can take corrective actions and 
continuously improve. It is no different from when we talk to 
companies who design automobiles, electronics, or banking 
services. 

When we look at it, it is easy to summarize. (See 
Exhibit E) There is planning, the up-front work you do to know 
who the customers are, what they need, and what they have done. 
The next step is quality control, getting rid of the glitches, 
the fires, the things that go wrong, and enabling people to fix 
the S6 • 

Then, the third area is quality improvement. Quality 
improvement is changing the level from wherever you are now to a 
new level that is far better than you ever dreamed. 

QUALITY PLANNING 

But let's start with quality planning. First, identify 
the customer. 

I was asked to be the discussant last year 1 at a special 
session in a conference in San Francisco on the teaching of 
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statistics in business schools. The session organizers and 
presenters were four professors who had done remarkable things 
on trying to improve the teaching in large classes, 200 to 500 
people in the class. These are required courses, usually at the 
freshman, sophomore, or junior level. These professors have 
been working for about five years on changing the jtructure and 
teaching methods. 

What they had done was truly remarkable. But when I 
listened carefully, one thing was missing. They had no idea 
whom it was they were teaching or who their customers were. 
When I asked the question, we got into a very lively debate. 
Are the customers the people who hire the business students at 
the bachelor's level? Are the customers the graduate schools, 
MBA schools that admit business students? Or other graduate 
schools, law or medicine, that some of the students are going 
to? Are the customers the students themselves? 

When we went through this interesting discussion of whom 
they are really teaching and whom they should be concerned 
about, it became very clear that we hadn't done the next step 
very well at all. This step is determining the needs. What do 
students really want? What do the employers really want? Has 
anyone from the university actually gone and talked to the 
employers who hired students five years ago to see how well the 
students are doing with the preparation given them? Or do they 

have to retrain them? 

One of the exercises I go through in ray class at 
Colombia University is the construction of a fishbone chart, a 
cause and effect diagram. 

We talk about what is the goal of the course. The 
students express it in their own way. The goal of the course, 
according to the students, is to maximize learning. That is 
what they are there for. They identify the different branches 
of the fishbone, the causes that drive the effect of maximum 
learning. They identify a large number of causes, but when we 
summarize and get rid of the redundancies, there are really six 
major areas: the teacher, the syllabus (what is going to be 
taught), the materials, the assignments, the facilities, and the 

students. . . 

Then the students break down these details. The rirst 
is what they want in the teacher. They are concerned about the 
quality of the lectures, whether they are interesting, whether 
they are relevant, and whether they are well prepared. They 
want the teacher to have relevant experience. They want the 
homework assignments to be relevant to the course. They want 
lectures and assignments related to the text. They go through 
great detail on what they expect from the teacher. 

On the materials, of course, they are interested first 
in the quality of the text. They are interested in the handouts 
and the visuals used in class. 

The syllabus they think is very important. Is it 
relevant to their needs? Is it relevant to other courses they 
are taking? Is it relevant to their degree? Is it relevant to 
the work they will do when they leave school? 
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About half my students, since it is a night course, are 
getting their masters whi.le they are working. They want to know 
if they can use tomorrow what they learn today. 

They also want the material to be new. They don't want 
it to be repetitive of things they already know. And they want 
it to be challenging. They don't want to be bored. 

And so they go through this in great detail, and then we 
start translating these needs into something I can work with. 
They translate these needs into what they think is important. 
They put the heaviest weight, in average 45 percent, on the 
teacher, on the presentation of the material. 

They put heavy weight, 30 percent, on what is taught, on 
the course, and the content. Third, and way down the list, down 
around 10 and 15 percent, are the materials. Of course, the one 
that I weigh higher than the students is the students. I think 
the students in the class, their participation, their 
preparation, whether they do the homework and whether they have 
read the materials is extremely important to the success of the 
class. They tend not to see that one as clearly as I do. 

The one they put very little weight on is the 
facilities: the classroom, the lighting, the air conditioning, 
the heating, the computing center, and their own study 
environment. It turns out they will put up with anything if the 
course content, the teacher, and the materials are good. 

But the facilities are where we put so much money in 
education. An^ we put so little in the content of the courses 
and the quality of the teachers. 

After we translate these needs we develop a measurement 
system, how the students are going to measure the quality, how 
they are going to provide feedback, what the quantitative 
measurements are, and how they can use them. So, rasically, I 
am teaching a course on quality control, and I am using the 
methods that we are going to learn in the class to start doing 
something, that is very important to them. The basic questions 
are simple. What do you expect? What do you think you are 
going to get out of this class? Are you going to have fun? Are 
you going to learn something? How is it going to relate to what 
you actually do? 

And so this class on quality control becomes real to 
them. This assignment is gone over in class as a group and then 
assigned to them in small groups of two or three. This 
assignment is always the best done of any assignment in the 
whole semester. The students work hard at this. Soma of them 
turn in ten and fifteen typed pages, when the requirement is 
only two or three. They turn in detailed charts, figures, 
measurement plans, and how they will take the surveys. They 
really get into it. The quality of education is important to 
the university students. It is important to all students. 

Let me give you an example on how this quality planning 
applies in industry, because there is a very good example of 
these first four steps. The Ford Motor Company years ago 
decided they had a problem. As they explain it, in 1979, 1980, 
and 1981 they lost $1 billion a year. They had a major 
management meeting and decided they couldn't keep doing that 
forever. They started something that is now called Team Taurus. 



And they started looking at very good cars. They did something 
ve-zy strange. They bought some competitors' products, and they 
actually measured them in the labs and on the test track. 

Now, the reason I say this is strange is that in 1980 
when Ford first started their quality program they came to Bell 
Labs where I was working and they were reviewing our quality 
system. We never laughed so hard at a group of visitors in all 
our lives. We waited until they left, but they were bad. When 
we went to lunch, they all wanted to ride with a colleague of 
mine because he had a Saab and none of them had ever ridden in a 
Saab. He let one of them drive it on the way back, and they 
almost didn't come back to the meeting because they couldn't 
believe the acceleration this four-cylinder engine could give. 
They couldn't believe the way this little car handled. 

Now, this is a car made by an aircraft company in Sweden 
noted for its technology, and none of these people had ridden in 
it. These executives were such high level they were 
chauffeur-driven in Lincolns to Ford every day. They didn't 
even ride in the cars that produced 98 percent of their profits. 
The chauffeurs took care of the cars during work, did all the 
maintenance, and if the car were really broken they had another 
car for them, so they never even knew if the cars worked or not. 
They were completely isolated from their own products, 
completely isolated from the competitors. 

That is why I find what they have done so remarkable. 
They took these competitive cars and they identified 428 quality 
dimensions. It is not easy. There are very many dimensions of 
quality for any product or service. Some measurements they 
could do on the test track: acceleration from zero to 60, miles 
per gallon, braking, skid pad resistance. 

Other measurements are more subjective. They brought in 
teams of customers. They selected the "best-in-class" ignition 
switch, customers think it is Toyota; best jack, customers think 
it is BMW; best jack storage, customers think it is BMW; best 
luxury door hand, they think it is Mercedes-Benz. 

They ranked these. They listtJ the winners in a book, 
and then they put a designer's name by each one and said, "beat 
it or copy it." And they gave a time deadline to the designer. 
They said, "Oh, by the way, we trust you, we love you, you have 
been working for us for a long time, but when you have your new 
design, bring it back, and we are going to bring the customers 
back and let them pick the winner. If it is not yours, you will 
have to get a license contract from other people." But 80 
percent of the time they thought they either beat the 
"best-in-class" or equalled it. 

As you probably know, in 1986 ford mad« more money than 
General Motors for the first time since 1924. Last year they 
were almost embarrassed by the $4.7 billion they made. They 
have gone into quality. It seems to be paying off. 

We have used many of the same methods in developing a 
product and developing product features in the development of 
courses at Bell Laboratories. The project management methods 
we use for developing a course are the same project management 
methods we use for developing a product. We go through careful 
specification reviews. We go through reviews with potential 
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customers, people who will be taking the course. We look at 
people who have taken similar things. We visit at least four or 
five other companies or universities who are teaching similar 
things. We characterize what they are doing. We bring in 
customers at periodic intervals — for example, ten people from 
manufacturing plants visited once a month to review the one 
course we were developing. We develop these product features 
very carefully with the customers step by step. And then we 
develop the process, the process on how we are going to teach 
the course. 

At :;he Juran Institute where we make our living teaching 
courses we have a very long qualifications rlan for every new 
instructor. Even though we hire senior people from companies 
who have been in this business or a long time, they go through 
at least a year of qualification. They're trained on certain 
parts of a course. Then they work with the key instructor. 
They go through about four or five courses with the key 
instructor, and then they are allowed to present the first one 
hour of the course, then two hours, then up to half a day, and 
then up to a day, always getting feedback from the key 
instructor, always getting feedback from the students. We 
always have a feedback sheet at the end of every course. 

Feedback from the students is one way of improving the 
process. But the students only know whether what they have been 
presented is good. They don't know what they haven't been 
presented. The things that are missing, the things that the 
students need that weren't even covered, cannot be rated by the 
students. So you have got to bring the people in from the 
outside. You have to also use some experts in the field to 
review the course. 

The next step is to transfer the process. How can we 
get the process that we have created, the course that we have 
created, into the hands of large numbers of instructors? The 
qualification procedure is one way. Standardization is another. 
We can create text materials, workbooks and visuals, but we find 
out that people don't use other people's examples very well. So 
we now have a training package to train instructors on how to 
create their own examples from their own experience and the 
experience of their companies so they can make examples come 
alive. 

A question came up earlier, does this work in the 
government? Can we really do these things? Our work on quality 
planning has been going through about a three-year cycle. We 
are actually videotaping our course this week and it will be 
available sometime in September. One of the best examples comes 
from the Bureau of Labor Statistics. One of the pilot sessions 
was in improving the accuracy of the Current Price Index. 

Dr. Janet Norwood, the Commissioner of the Bureau of 
Labor Statistics, is very much leading their quality improvement 
effort. They have excellent leadership at the top. She is 
doing many things personally. It really does work in the 
government. There are more and more eood examples. The 
government of the state of Wisconsin and the Madison, Wisconsin 
mayor's office have done some remarkable things. The Bureau of 
Labor Statistics and the Internal Revenue Service have done 



excellent work in quality improvement. The IRS, led by Mr. 
Larry Gibbs, the Commissioner, has reduced data entry by 
billions of keystrokes a year. 

QUALITY CONTROL 

I will just very briefly mention quality control. You 
have to have some way for people to know what quality is. 
Individuals have to have a clear definition of what quality is, 
what is expected of them, and a sensor, a way to measure what 
they are delivering. Individuals must have a way to analyze 
this and compare it to the standard. They must have a way to 
either correct the problem or to go back and improve the product 
and the process. 

If you get this down to the individual level, to the 
teacher, the administrator, the person running the program, then 
all sorts of wonderful things happen. 

QUALITY IMPROVEMENT 

So now if we look at quality improvement, the last part 
of the trilogy, we just see a very simple sequence. You have to 
know you need to change. You have to identify the top 
priorities. You have to have some organization to guide the 
projects and somebody to actually do the work. The guiders are 
usually managers, and the workers are usually the managers and 
the work force members. 

You have to have a breakthrough in knowledge. You have 
to understand something you didn't before. You have to have a 
breakthrough in results. You have to have the breakthrough in 
cultural resistance, and then you have to be able to control at 
the new level. 

In thinking about this waste we are trying to eliminate, 
we see there are two major wastes. One is the waste of the 
students, teaching the wrong thing. I tried to address that 
with the examples on planning. That is the key waste. There 
is nothing worse than teaching them something they don't need. 

Not long ago IBM did a study of 5,000 engineers. Two 
had used differential equations in the first two years of 
employment. All had designed and run an experiment. All had 
been trained in differential equations. None had been trained 
in experimental design. 

The other waste is teaching things wrong. And that is 
the efficiency issue. That is not using available material. 
That is re-inventing wheels. That is not taking advantage of 
new techniques: video, computers, physical models, simulation, 
other aids. It is failure to integrate things being taught, for 
example, having what the mathematics people are learning tied to 
the physics they are learning, tied to the chemistry they are 
learning. Instead, these things are often taught in different 
years, much less in different classrooms. 

There are other wastes. Failures to realize related 
needs of students. For example, teaching physics students 
German by reading biology papers. Now, the ultimate counter 
example of that, I think, has happened this summer at MIT where 
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they have a special course in teaching technical Japanese to 
computer scientists and electrical engineers. People who 
alreauy speak fluent Japanese and have an eighth grade knowledge 
of Japanese are coming in on an NSF sponsored course to learn to 
read Japanese reports in Japanese. These reports are just in 
computer science, just in electrical engineering. America is 
trying to catch up in these two very vital fields. 

Another waste is using unrelated examples. Ernst 
Rothkopf of Colombia University and Bell Laboratories did an 
experiment at Texas A&M. They ran one course with 60 students, 
teaching statistics with the traditional examples, with cards 
and dice. They ran another course where they used manufacturing 
examples, where they related the problems to more engineering 
type things. And then they went back and tested these students 
six months later. The ones who had learned on cards and dice 
were very able *:o solve card and dice problems six months later, 
had never appli I what they had learned to their work and 
couldn't apply it on the simple test questions. On the other 
hand, those who had learned statistics in a more manufacturing 
engineering way had used it in their work, were able to apply it 
to those things, but couldn't solve the card and dice problems. 
People remember the examples and they tie the theory very 
carefully to the example. 

Another way we fail with students is by failing to 
support their applications. We let students go off 
half-learning things, try something and fail and never try 
again. 

And, of course, there are other wastes—wasting the 
teacher's time, teaching the wrong student, in one of my 
classes in Vietnam in inventory control we looked up and saw 
everybody had an MP armband. Eighteen students, all MPs. We 
didn't believe they had that many people in supply. We asked. 
They said, well, everybody else is out fighting. The post 
commander said we had to have 18 people for the class, and we 
are the only 18 available. 

Using poor materials, making do, redevelopment and 
reinvention are all wastes. When we prepared one course at Bell 
Labs, we had 800 hours of preparation time for every hour of 
course time. This included developing the software, developing 
the course, improving the course. When I think of the time that 
I and my co-authors spent on our book, it is 200 hours of 
preparation time for every classroom hour. There is phenomenal 
preparation time to do things right. You can't keep doing that 
over and over, instructor after instructor. You have got to 
take the best, coordinate it, use it so that everyone can use 
the best n.ethods from the best teachers, and incorporate them in 
their classes, and make good stuff widely available. 

And, of course, appraisal is all waste. All tests and 
examinations are a waste. If the students knew everything and 
we knew they knew everything, we wouldn't have to do any 
testing. And rework is waste. Every time we have to reteach 
because the students didn't understand, because one student 
didn't understand or because a group of students didn't 
understand is waste. 




Preparing materials for class the second or third time 
when nothing has changed is waste. Explaining assignments that 
should have been done right because the student didn't 
understand the assignment is waste. Teaching material that 
students should have learned somewhere else is waste. Most of 
the teaching we do in industry is waste. It should have been 
done in the university. It should have b' in done in the high 
schools. 

When we are teaching factory workers basic 
problem-solving, it is waste. We are doing it because we have 
to survive, not because that is what we should be doing. Dave 
Kerns, chairman of Xerox, says the Achilles heel of America's 
ability to compete is the quality of education American students 
get in high school. When he said that, I challenged him 
afterwards. We had a little bit of an argument. I think the 
engineering training is even worse, and that that is the Number 
One problem, but I don't disagree with him that the high school 
education is almost worthless. 

The American Assembly closed their report with a very 
strong statement: "Education is the Number One priority in 
restoring America's competitiveness." 

Thanks. 



ii 



Mr. Caplan: 

Thank you very much, Blan. 

We are right up to the time to break for lunch. If 
there is one urgent question I will accept it. If not, you can 
wait until the panel and get your questions in of Blan. 
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AFTERNOON SESSION 



Mr. Caplan: 

Well, if we can reconvene, we have now recovered from 
our equipment problem, so considering that quality is the 
subject matter of this seminar, I think we have illustrated to 
you, not necessarily by intent, the concerns for quality in our 
society that we have been trying to emphasize to you in this 
discussion, and that being the case, we want to address a 
specialized branch, a somewhat different industry now. 

Bill Golomski made the point earlier, and Blan echoed 
it, that there "ain't no such thing as a different industry," 
but in fact there are some things about the software business 
which render it a bit unique, and to address those subjects we 
have asked Dr. Basili, who is chairman of the Computer Sciences 
Department at the University of Maryland, to talk to you a 
little bit about software quality fundamentals. Vic. 



Presentation; Software Quality Fundamentals 



Dr. Basili: 

I agree with what has been said so far today. What I am 
going to do is to try to map some of the concepts onto the area 
of software development and tell you about specific problems in 
the areas with regard to achieving quality. 

I will begin with four issues that I believe are key 
problems with regard to software quality. Most of them have 
already, been mentioned, but I am going to give you maybe a 
slightly different view. 

The first issue is that we need to define quality 
operationally, relative to the project and the organization. If 
I am going to build software, I've got to define what I mean by 
quality and that turns out to be an absolutely non-trivial 
process. In fact, more importantly, I have to define quality 
operationally, which means in a measurable way. I have to 
define it in such a way that I can use it to motivate what I 
mean by software quality, so that I can use that definition of 
quality to evaluate software, i.e., so that when I am done I 
know I got it or I didn't get it. Then I can use that 
definition to build models of the software process and product, 
so I can do prediction. 

The second issue is the recognition of the intricate 
relationship between quality and the process. Software is a 
very primitive field in a lot of ways. We are learning about 
methods for developing software, and we are not very far 
advanced. But it is clear, given a set of goals for software, 
then I have to find the processes and methods I am going to use 
to develop that software to meet those goals. 
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The third issue is find mechanisms for evaluating 
software. How do you do evaluation? The first process step is 
definition. That is hard, but it is not nearly as hard as 
evaluation, because evaluation has to be done in a context, and 
I have to make sure that the scenario and the contex; in which I 
am going to do the evaluation is correct. 

And then the last issue is to create an organizational 
scheme so that I can deal with evaluation within that 

organization. 

Why are these four preceeding issues problems for 
software? Why is defining quality, for example, a particular 
problem for software? I will give you three reasons predicated 
on the difference between software development and hardware 
manufacturing. They are: the nature of the process, the 
difficulty of what we are trying to accomplish and our lack of 
understanding of the process and product, and the fuzzy 
expectations for outcomes. 

And the first one says that software is different. This 
is development versus the production process. So, for example, 
if I were to think about this in an engineering sense, it is not 
a production line where I am building a whole series of widgits. 
It would be taking the engineering process and trying to measure 
the process of creating the first prototype of the series of 
things I am building. It is the major difference between 
hardware manufacturing versus software manufacturing. It is a 
major difference . 

Also, software is not visible, which turns out to be a 
very dangerous thing tor software, because it gives you a 
strange feeling about what it really is. So some of the 
traditional ways that we would view hardware manufacturing 
because we can see the product are different from how we would 
view software. It isn't really production in the normal sense 
of the word, and it involves a non-visible product. 

The second major point is that software is difficult. 
That may sound like a strange statement to make, and this is 
something I would say internally to the software business people 
who don't understand that necessarily. That is, there has been 
a feeling in the business that software really isn't very hard, 
that it is easy to build software systems:, and I am going to 
argue that that is absolutely, positively, unquestionably not 
true. 

There is a complexity issue that we get into when we are 
building software that is a little hard to understand. Let me 
give you a couple of examples of what I mean by that. In the 
first place, it is one thing to write a 20-line program.. 
Anybody can write a 20-line program. We teach this to kids in 
high school. That is different from what we mean by software 
development: building a production system that is large, that 
has all kinds of complexities associated, that is integrated 
with other pieces of the system, that is well documented, that 
has all the things I need. That makes it a very hard problem. 

Worse than that, there is a lack of understanding that 
it is a hard problem. My favorite story about that occurred 
when I was teaching a freshmen r-ourse in computer science at the 
University of Maryland. Our cc se is very mathematically 
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oriented, and it is a tough course. We build mathematical 
models of programs and teach students how to reason about 
programs using mathematical models. A student came in to me, in 
the first semester, after the first exam, and had gotten a D on 
that first exam. He was very upset and said, "I can't 
understand why I got a D." So we talked about vhat he did 
understand and what he didn't understand. Halfvay through the 
conversation the student said to me, "I know I «m a first class 
computer scientist." And I thought to myself, if I were a 
chemistry professor and this kid came in, first semester of his 
freshman year in college and said to me, "I know I'm a first 
class chemist", I would have just laughed. But he said computer 
scientist, and so it is not so funny. Is computer science a 
science? Is it an engineering field? And the answer is, yes, 
it is I And just because someone understands how to use a 
Gilbert chemistry set at home doesn't make them a first class 
chemist. Understanding how to program at home doesn't make 
somebody a first c]*ss computer scientist. 

There really is a technical discipline, an engineering 
discipline. It is very complicated. And the problem is, as 
Rodney Dangerfield would say, software engineering doesn't get 
any respect. It is not understood. And I don't mean just by 
the outside world. Most managers don't understand the 
complexity of the thing they are dealing with, and that is one 
of our problems. 

Software is difficult for a number of reasons. First of 
all, what is the software? We are building a system. What is 
the software? It is the part of the system we least understand. 
That is why it is software. If we really understood it well and 
it was organized, then it would be part of the hardware. 

Second, we are always doing something new, so first of 
all we have trouble estimating. If we have done it enough times 
then we can turn it into the hardware, but we are always 
building something new, and that is complicated. 

Third, we have a lack of models, especially tractable 
models, for almost any product or process with which we are 
dealing. That makes it hard. 

And last but not least is, software (whoever gave it the 
name "soft" did us a disservice) has a constant requirement for 
change, and we really do not understand the implication of 
changing a complex system, or what happens in terms of the 
entropy or the unstructuredness of that system as it evolves and 
changes. 

The third major problem we have is fuzzy expectations. 
Given what I just said about software being difficult, then the 
software development process is fuzzy. We have fuzzy 
expectations in terms of poorly defined methods, usually very 
poorly defined requirements. We must interact with the users. 
Who is the user? How well do we understand what their needs 
are? Very often we build something because it seems like a neat 
idea. And worse than that, in many cases, we have reached a 
level where people accept poor products. They accept poor 
quality. We have made the public believe it is okay to have 
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poor quality. How many times have you talked to someone on the 
phone and they say, well, it was the computer. Oh, that's okay, 
if it was a computer error. We know they make mistakes. 

So that is the nature of the business we are dealing 

with. n , 

I would like to discuss a couple of key issues. One of 
them is, we * '*e a standard process we talk about when we build 
software, ar ,iere is a standard, what we call a software life 
cycle. Thif is an example of something we don't fully 
understand. We view it as starting with some kind of 
requirement: here's what the problem is. And then we carefully 
specify in some notation a specification. Then we design a 
solution, code it, and then test and evaluate it. That is how 
we say we build software. 

The problem is that we always talk about the process, 
not the documents. That is the first thing I think we do wrong, 
because there are lots of process models. The minute you 
standardize on a process model, you are in trouble. We need to 
standardize on the set of documents that you are creating, and 
then you can find lots of process models, depending on what 
qualities you want those documents to have, or what you want the 
product to be. This view gives the flexibility to manipulate 
process and product together. 

One of the things that you heard both speakers talk 
about this morning was the relation ig of process and product 
and the need to manipulate the process based on the product. 
That is something in the software business we are only now 
beginning to understand. 

What is wrong with the way we interpret the software 
life cycle? Let me just give you a couple of problems. The 
first one is, again, those processes; what are the basic 
technologies in the software business? I am going to tell you 
that one of those basic technologies we have in the software 
business is reading. 

I write a requirements document and someone has to read 
it. If I write a specification document, someone has to read 
it. The same with a design document and a code document. We 
used to think, years ago, that writing programs meant we were 
communicating only with a machine. That is not true at all. It 
is sort of a side effect. We are really communicating with 
people who have to maintain the system, use it, understand it, 
modify it, enhance it, et cetera. So, how much training do we 
do in what I would call the most basic technology of software, 
which is reading? Do we teach people to read? 

How many people have had a course in programming? Okay. 
So you can answer this question. How much time dV you spend 
reading as opposed to writing? Okay. Zero. Suppose I said to 
you, we are going to teach everybody a foreign language. Okay, 
everybody, start writing. Don't read anything. Just keep 
writing, and eventually you will get there. And then we wonder 
about this issue of quality. How do you define quality in the 
first place? By good writing. What is your model for writing? 
We know from an educational point of view the best model for 
writing is reading. And if we have never learned to read, wow, 
are we in trouble! 
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So in a sense what we have in the software business is a 
large number of what I would call software illiterates, people 
who can write but they can't read. And so that is one of the 
first fundamental problems we have with software quality. And 
part of it is an educational issue. We don't do the right kinds 
of training. We don't do the right kind of education. In fact, 
it is more complicated than that. I read very differently for 
different environments. People write for an audience and I want 
to read that way. There are lots of ways, for example, of 
reading a program. I can read a program sequentially. I could 
read it by abstracting function, then recognizing it in 
hierarchies. I can read it by tracing paths, because there is 
control flow in this program. Which techniques do I use under 
the circumstances? 

I read from a perspective. I am going to read a 
requirements document. We have already talked about who is the 
producer, who is the consumer. We have a model we talk about, 
the producer-consumer model. Who is the producer, the person 
creating this document? We talked about the life cycle being a 
series of documents. I produce this document for an audience, a 
consumer. Who is the consumer and what is his or her 
perspective? How do I read a requirements document, for 
example, from the point of view of the developer who is going to 
develop the system, versus the maintainer of that system, versus 
the user of that system? And are those reviews done? What 
qualities doss each look for? Not only do we not know how to 
read, we don't even have good methods. We have not been trained 
for that reading. The processes aren't well understood because 
we don't even understand the basic technologies, we don't 
educate and train for them, we don't even know how to combine 
them into a larger process. Reading is a technology that needy 
to be combined into a method to make it usable and manageable 
and measurable. How do I do that? And how are the technologies 
and methods changed for the environment? Another problem is 
that all development environments are different. I don't have a 
standard manufacturing plan. It always looks a little 
different. People have different backgrounds, different 
equipment, different needs, and maybe different quality goals 
for different classes of products. 

This leads to another problem we have in the software 
business. People really believe that software is software is 
software. That is, there is a belief that all software is 
similar. That is a dangerous thing. It would be like saying 
hardware is hardware. It doesn't matter whether I am building 
satellites or toasters. I'd use the same methods, the same 
tools, the same training. Well, that's absurd. It is also 
absurd in the software business. Do I train those people 
differently? Do I use different methods? Do I adapt them 
differently in different environments? 

I started off this talk saying that I was going to talk 
about the software business and software quality issues. So 
where am I going? I am trying to argue that the quality 
concepts that you heard this morning are valid for the software 
business- The problem is, the software business isn't organized 
for those quality concepts. 
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What I mean is, we have to re-understand and re-think 
the nature of the software business in a way that makes it work 
in a quality oriented environment, because everything that was 
said this morning is right. And the nature of what we are 
tr ying to do in the software business is wrong . We scaled up a 
business that is only 30 years old (think about it, 30, 35 years 
old) and we scaled it up all wrong, so it wasn't geared to 
quality. Quality was never the driver. Innovation was. 
Someone who could build something and make it run for a short 
time, that was the winner, that was the person who got rewarded, 
that was the organization that got rewarded. And now we are 
discovering, much to our chagrin, that you can only do that for 
something so big and for so long a period of time and maintain 
that for such a time and then it comes crashing down around you. 

It has taken a long time to convince the people in our 
business, I mean the software business, that they are in 
trouble. They are looking at the wrong things and that they are 
on a path which isn't going to take them up slowly, it is 
limited. It is taking them down (I am thinking about Blan's 
chart) because they don't understand as they are hitting new 
strides and complexities. The approach they are using which 
says, "stand on the floor and jump as high as you can", won't 
work. The approach that has to be brought to bear is one that 
says learn, learn fast. Le?rn how to train and educate people 
how to do it better the next time. 

Several professors at the University of Maryland did a 
study that was supported by IBM a few years ago, and we looked 
at U.S. and Japanese companies in the software business. What 
we noticed was that American companies were not doing 
post-mortems. They would build a product, and not ask what they 
did right or what they did wrong. What they did was hu.cry on to 
the next project. There was not only no feedback, but V.here was 
no learning process. Now, I don't mean to imply that feedback 
and learning are easy. They are not. Part of the problem with 
feedback is you have to know what you want fed back. Ani when 
we are in a primitive technology, it is hard to know whac to 
feed back. One of the things we need to talk about is, how do 
you set goals and how do you turn them into measurements based 
on models oi the processes, products, and people and how do you 
learn what you've done right and wrong and where do you nake 
improvements? That is Number One. 

The second issue is that the life cyc.Te view we just 
discussed has major pieces missing. First, how was the user 
represented? Indirectly through a requirements document. 
Somehow there was a process carried out that made the user 
understand that what we wanted to build was not represented in 
in the process model explicitly. 

The process model was very intro ->ective. The process 
model said: "assuming I know what the ^quirements are, what do 
I do?" It was written from a point of view of the developer, 
not the point of view of the user. 
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Second, where and how is quality control represented in 
that process? Where were the checks? And the last issue is 
that process and product qualities are poorly articulated. What 
are the goals of the process and product, and how are those 
goals tracked through the life cycle? 

Let me give you some general goals for software 
development. The first one is a goal written from the point of 
view of the manager. The manager's goal is to develop a set of 
documents, e.g., requirements, design, code and test documents, 
that all represent the same system at some point in time. That 
sounds like a very simple-minded goal. What I just said sounds 
obvious except, if you know anything about the software 
industry, that never happens. The requirements exist, and then 
they get changed, but no one updates them, and before you know 
it the only thing that represents what the system really does is 
the code that runs on some machine. And so someone has got to 
go ahead and maintain it when they don't even know what it was 
supposed to do in the first place. 

A second goal, from a user perspective, is to develop a 
system that satisfies the user needs with respect to 
functionality, quality, cost, et cetera. How is that checked? 
How is thao guaranteed? Where was the user in that process to 
begin with? 

If the user is an educational organization that wants 
educational support systems, how do they make their needs known? 
How do educators get the kind of software that they want built? 
Who is asking them what they want? How do they know what it is? 
And how do you get it so it is a quality product? Who is 
articulating the needs of the educator? users must carefully 
articulate their needs and interact with the developer to assure 
their needs are understood and the feasibility of their needs 
evaluated. 

Another goal is from an organizational perspective. If 
I built the system once, I should be a lot smarter. The goal of 
any organization once they build something is that they should 
get smarter, and the next system should be better and cheaper. 
This is impossible if there is no learning and feedback process 
involved. That, in my mind, is a classic problem that we have 
in the software business, that we are not learning from what it 
is we have done before. 

I said that goals for software quality weren't carefully 
articulated. I want to give you an example of what I mean by 
that. If we are writing a requirements document, what are the 
goals of that activity? What are the goals of the requirements 
document? How would I specify the criteria by which I am going 
to evaluate whether I have done the right thing? The benefit of 
stating those criteria is that they not only allow me to 
evaluate, they allow me to tell the person who is writing that 
document what I expect them to do and how it is supposed to 
happen. If I think about what the goals of the requirements 
activity are, which are to try and capture what it is the user 
really wants in the system, then what am I doing to make that 
happen? What does it say about user input, for example? From a 
developer's point of view, do we send people to the customer to 
see how they do but iness? Do I go there and watch the nature of 
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the business and say, ah, that is the way you do business so 
what you really would like is something that does this? Do we 
get the user involved in requirements definitions and evaluating 
prototypes or even testing the system? 

Do we create models of how the customer uses that system 
so that we understand how it is going to be used and how it 
could be user-friendly? And if the answer is, we don't, then we 
have lo ;t quality from the beginning. 

I want to raise that life cycle process model one level 
and talk about what I will call the quality-oriented software 
life cycle. Step One. The first thing I want to do is 
characterize my environment. That is, describe the environment 
in which you are working, understand it quantitatively, that is, 
what kinds of errors do I make? What is my application? What 
are my resources? What kinds of equipment do I have? What 
class of product is this? Which methods am I using as part of 
my process to develop this software so that I can understand it 
better? 

Step Two. Once I characterize and understand my 
environment, I set my goals. Here is what I want to happen. I 
might want certain characteristics for the software, a certain 
level of reliability, a certain aspect of user friendliness. To 
help set these goals we have a paradigm we use, called the goal 
question metric paradigm, in which I take the goals, generate 
questions, and then come out with the actual measures based on 
what it is I am trying to make happen so that I know whether or 
not I have achieved my goals. I can't just say I want it to be 
user friendly. I have to tell you exactly what I mean by that 
in a measurable way. 

Step Three. What are the processes I am going to use to 
achieve my goals? I might choose methods or tools or techniques 
or processes and combine them in different ways. I may iterate 
through these first three steps. If I were to map this into 
what Blan talked about earlier, this is the planning stage. 
These are the things I have to do in planning, and it dominates 
everything I do. When I do planning, I certainly want to reuse. 
I want to learn from prior experiences. For example, I don't 
want to recreate measures over and over again. The assumption 
is, I have created a data base of measures and I should reuse 
that data for evaluation and building models. 

Step Four. Then I go through the process of actually 
building the system, but building the system driven by those 
goals, using those methods and evaluating, not just the 
products, but also the processes that I am using to make sure 
they are the right processes, providing feedback in real time. 
That issue was raised this morning. It is an absolutely 
critical thing. I want to know as soon as I have done something 
whether I have done it right or not. 

Step Five. Finally, I want to do a real analysis, both 
in real time and as a post-mortem at the end of the project. I 
want to write lessons learned documents and incorporate 
information I have gained from building the system into my 
environment so it becomes part of the characteristics of my 
environment. Then, when I start all over again, I am smarter 
than I was the first time that I went through that process. 
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This is different than what we described as the standard 
life cycle. First of all, it raises the level. Step Four in 
the quality improvement paradigm allowed for any process model 
for development. When I plan software I have to think about 
choosing the appropriate model for development. 

Second, it is not a bottom up process, it is a top down 
process and requires top level commitment. There was a 
discussion this morning about commitment from the top. 
Everybody has to agree this is the way it happens. There is no 
choice. It has to permeate from the top. 

Third, you don't evolve into this paradigm. It is a 
revolutionary process. That is, to get from where we are, at 
least in the software business, to what this paradigm implies is 
a revolutionary process. That is the bad news. It is not 
going to happen on its own. Someone must make it happen and 
that is why the top level commitment is really important. It is 
revolutionary because we need to think differently. V 7 e need to 
train people to think differently. 

The quality improvement paradigm is just the scientific 
method applied to software. I must understand the domain, 
establish some sort of hypotheses, which I called goals and 
questions, the things you want to understand, go through the 
process of measuring and evaluating using the appropriate 
procedures, decide what it is you have gotten, learn something 
and start all over again. The object we are looking at isn't 
something in nature. It happens to be an artifact called the 
software product. But that is what we are doing. We didn't 
invent something brand new. It is just the scientific method 
applied to software development. And that is what quality is 
about. I am trying to understand what is right, build models of 
understanding . 

So we need to think differently about software, we need 
feedback, we need to learn, we need to test hypotheses as we go 
through development. That is really what we are talking about. 
Maybe the words sound different, but they are the same things I 
heard this morning. The process can be achieved in small steps. 
The nice thing about it is an organization can start a certain 
project or series of projects in this way to learn how to apply 
the method. There is a learning process associated with 
application of the method that we ar* talking about here. 

But the good news is, once yc are in that paradigm, the 
later stages are evolutionary. Once I am measuring what I am 
doing, gaining feedback from it, and learning, I can improve a 
step at a time. But I can't do that until I put myself in the 
context of the paradigm. I can't learn from the way we are 
doing software development now. We are not learning, at least 
not learning fast enough. The people who are learning are the 
managers of the people who are on that project. They are not 
recording that knowledge, that information, for the organization 
to have or to use in any way, shape, or form. 

In conclusion, the basic ideas are to define quality 
operationally, through measurement. There are lots of 
definitions of quality, and they not only vary from field to 
field, but they are going to vary within fields and within 
organizations. I want to evaluate the effects of methods, 
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tools, and processes. How well do they work? I am measuring 
not just the product, I am always measuring the process. I want 
to organize for quality assurance and control by characterizing, 
planning, measuring, evaluating, and feeding back and improving. 

The main issue here is that measurement is used 
actively. The key issue about measurement as far as we are 
concerned is it is the enabler of the learning process. Learn 
about the process and product. And improve the process and 
product in the appropriate context. There was a statement this 
morning that every job has two parts to it. One is the doing, 
and the other is looking at how to improve it. I think that is 
exactly right. 

Thank you. 
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Mr. Walser: 



Thank you very much, Vic. 

One of the sad things is that there were a lot of papers 
in his presentation that he didn't even get to, so maybe some 
other time we can get some more of that information also. 

At this time we are going to do a quick change. We 
would like to have the presenters of the panel come up to the 
table. And while we are doing this, if the people on the next 
panel, on the wrap-up panel, would just visit briefly with Frank 
Caplan out in the foyer and organize yourselves, then we will 
just move right on through our agenda and we will get back on 
our time schedule. I think we will be able to handle it in time. 

One request that I would make of you is please use the 
documents fo*. your written questions. So, come on up, 31an, and 
take your seat, and Bill. Vic's coming in a minute. 

The microphones are also available for you to use. Use 
them. Questions and answers will begin in a moment. 



Question and Answer Session 
for Presenters' Panel 



Mr. Walser: 

The first question that I have was to Bill Golooiski , 
from this morning's session. Did you suggest that leadership in 
change and improvement of quality necessarily comes from outside 
the organization? 

Mr. Golomski: 

No, I did not suggest that leadership only comes from 
outside the organization, but I did suggest that that is one of 
the places that you should also look for it, because you can get 
inbred within a given discipline or within a given organization. 
I think it is well to have people from outside come in, also to 
read journals from outside your field from time to time, to see 
whether or not there is anything that might be similar to what 
you are working on. 

Mr. Walser: 

Vic, I noticed you smiled. Do you want to add to that? 
Dr. Basili: 

No, that was just a little comment on my saying I 
rediscovered a field without looking on the outside, so it is a 
really important thing. It sounds self-serving when you say 
that, but I believe it is absolutely correct. There's two 
things. One thing is, if someone does see the right kinds of 
things inside, no one tends to listen to them because they ace 
inside the organization. That's Number One. Everybody knows 
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that. And the second thing is that very often you do get a 
jaundiced view inside an organization. You have been seeing 
things in a certain way, and then when someone comes in from 
outside they can say, well, you guys just didn't turn the light 
on or no one has noticed what was going on in that room. That 
is just a common sort of thing. 

Dr. Godfrey: 

Let me add one thing. There is another place that 
leadership comes from outside, and that is from the customers. 
I think we all get very shocked when we really find out what the 
customers think of our work, and that can be outside or it can 
be inside, as Bill made very clear today, with our internal 
processes. The next person in line may have a very different 
view of your quality than you have of your own quality. 

Mr. Walser: 

The next question is, why do other nations such as Japan 
seem to have a better reputation for quality products and 
services? I think that means than us, the United States. 

Dr. Godfrey: 

Let me start. I will start, because that's easy. It's 
because their products are better. I mean, let's face . 
U.S.A. Today , on the cover, has the ranking of troubles per car, 
and if you look at the first seven, they are all foreign cars, 
either West German or Japanese. If you look at Consumer 
Electronics, most products that we can buy for home, stereo, or 
even color television now aren't even made in the United States 
because we haven't been able to keep it up. 

It's based on fact and the fact has carried over to 
perception. A few years ago, while RCA still made consumer 
electronics in the United States (as you know, they are now a 
French consumer electronics company) , they did a study where 
they took a Sanyo VCR and they put an RCA name on some and left 
the Sanyo name on the others. They did this on a university 
campus, and they had over 900 students and faculty members judge 
their relative quality. Over 90 percent said that the Sanyo was 
better and they were identical products. The perception of 
Japan being better than American is so firmly ingrained that 
even when they are identical products people believe that. 

And the truth is, and we have to face it, in many 
products, and the problem is it is the most visible products, 
it's the products we have in our home like the consumer 
electronics, it's the products we drive. It's not the products 
we don't see so carefully, like medical equipment, like some of 
the insides of other products. But in most of those products we 
have lost the leadership. It has been recognized by the public 
and Americans are having to make a big fight back. And it is 
going to take awhile. In some products we have done wonders, 
but we can't convince the Japanese to quit improving. You know, 
if they'd just get fat, dumb, and happy we'd have a chance. But 
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now we have countries like Singapore, which has a national goal 
to educate 25 percent of the population in quality improvement 
and productivity in the next three years, driven by the National 
Productivity Board, which is a huge government complex with the 
most beautiful educational facilities I have seen anywhere in 
the world. 

When you have countries like that — some American 
industries call them the five tigers; in Korea, where education 
is still Number One, the top status is still a professor or an 
educator; where the rewards that AT&T Gold Star gives employees 
in their factory for doing good work are free education courses 
at night, and people are honored to be given an extra four hours 
every Thursday night in technical training, rather than cash 
awards— we have some tough competition and will continue to have 
tough competition for a very long time. 

Mr. Golomski: 

Blan has put this in very strong perspective for us, and 
I think one of the things, if you were to do a sociological 
study on, say, prestige associated with professions in the U.S., 
we would find that the educator has dropped relative to that 
which we find in other countries. And of course the question 
is, is that cause or effect? 

There are a few industries in which the U.S. still has 
the world class standard, and of course they are working very 
hard to make sure they maintain that. Many companies are 
especially known throughout the world, here and there they might 
have some products that are being attacked, but I think if you 
were to take a company like, 'ay, the Eastman Kodak company, 
whose chairman is chairman c lational Quality Month this 
October, that a goodly numbet of his products are world class. 
Some are being attacked by some Japanese companies, and inroads 

are being made. 

And by the way, I would recommend to you that you 
consider in your various agencies and organizations to be a part 
of National Quality Month this October, to make that a special 
initiative. Get together with the rest of the nation and I 
think it will help you, and it will help all those with whom you 
come in touch. 

Mr. Walser: 

Another question, and this one is, Mr. Golomski, 
probably directed to you. Don't you think you are downplaying 
substantially the front-end investment needed to assure a 
successful quality improvement program, and also, how do you 
reconcile short-term goals, i.e., short tenure at high 
government levels and productivity, with longer term quality 
improvement goals? 

Mr. Golomski: 

Blan, why don't you answer first. You are experienced 
in education. 
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Dr. Godfrey: 



Bill is going cO let me start so he can think some more. 

Yes, there is an up front investment, And those of us 
in the business will say that quality is about the best 
investment money can buy. But we won't hide the fact that it is 
a real investment, and when you start talking about training the 
numbers of people that some companies are training, that is a 
phenomenally expensive investment. 

IBM estimated last year that they spent over $1 billion 
in training. AT&T estimated they spent over $300 million in 
training. Xerox has very similar numbers. Phenomenal 
investments in keeping people up to date on the technology, 
keeping people up to date on the new quality methods. With the 
short term look, I think most companies that are really 
interested in maximizing quarterly profits, we don't have to 
worry about too much. They are going fast. 

The ones that are building for the future are the ones 
that are talking five-year, ten-year plans. The ones that are 
talking three to five years just in the training plan for the 
basic fundamentals of managing quality. I think you are seeing 
more and more top companies, and Bill listed some of them, 
American companies who are still above the curve, maybe still 
having some change of leadership, and they are fighting very, 
very hard. 

You are seeing more of that view. I don't think we have 
it nearly as solid as the Japanese. We don't have it as solid 
as the Europeans. I was just party to p. discussion in Europe 
where they are talking about training the top managers, the 
people that are going to be managing directors of some top 
European electronics companies. The training program they are 
laying out is two weeks a year for five years. I haven't seen 
Americans thinking that way yet. But I think that will change 
very, "ery rapidly. 

Mr. Golomski: 

I think the whole matter of cost is always relative, and 
it is relative to the benefits that you get, and in 
organizations there are several ways to measure the benefits. 
One is in terms of employee satisfaction. Now, does that affect 
the bottom line? Well, it might be difficult to measure, Maybe 
it will reduce the turnover of employees. 

Secondly, your customer satisfaction. Again, even in 
government enterprises your constituents are very important. 
Greater satisfaction there. Then we can talk about the usual 
ones we talk about in industry, cost reduction. That will apply 
to you as well. The question is whether it will apply in the 
first year enough to offset the cost of education and training 
or not. It all depends on how rapidly you go through the 
organization. 

One of the things I like to do is not move through an 
organization rapidly, but to move to a given level and then hold 
until there is complete buy-in in some kind of activity. When 
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you have that, move to the next level. It takes a little 
longer, but your goal then isn't educate everyone, but simply 
to get some results as a result of the education. 

Now the last, which governmental groups probably are not 
concerned with to the degree that industrial companies are. 
Industrial companies are interested in revenue enhancement. I 
don't know what that means to you, if that means a larger budget 
or what. But anyway, those are the criteria. Those four 
criteria are the normal ones that are used to measure the 
benefits, and for the most part industrial programs will find 
that within the second year, sometimes within the first, you are 
getting a payback. 

Mr. Walser: 

Another question is, what have the quality sciences done 
with respect to the assessment of quality in the work of 
agencies or organizations providing elementary or secondary 
education, and with what result? 

Mr. Golomski: 

None, to the best of my knowledge on any broad scale. I 
think there are little activities here and there in elementary 
and secondary education. Some school systems are trying this 
out. As I mentioned, the National Council of Teachers of 
Mathematics has two programs, one establishing standards 
according to various grades for content, and then secondly 
trying to have a statistical literacy program brought into the 
schools . 

Mr. Caplan: 

I can add one point to that that this gentleman may not 
be aware of. The state of New York Regents Board has, I guest, 
about five years ago, informed the school systems in New York 
state that they were going to institute questions on the regency 
examinations associated with probability and problem-solving, 
starting with the first six grades and then moving grade by 
grade up through high school. That process has started, and I 
don't know what the results are, but the measurements are 
beginning to be seen in a limited portion of the quality 
sciences. Recognize that this is just one relatively small part 
of the total, but at least that effort is taking place. 

Dr. Godfrey: 

Let me add one thing. That was news to me, too. 
Thanks, Frank. And I know of very few examples of anyone doing 
anything, but one of the interesting things was, in a poll of 
high school students of things that they would be interested in 
knowing how they are done and the methods behind them, of the 
top three — you will have to forgive me, I have forgotten the 
third — but the other two were polls themselves; how can Gallup 
take 2,000 people during the Presidential election and predict 
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within one percent who is going to win and so forth? That 
fascinates students, and they are very curious about how this is 
done. And the other, believe it or not, was quality control. 
How do companies actually make things that work were things that 
came up. And speaking of needs or wants that were not 
addressed, we are actually getting requests from our customers 
to start. 

Mr. Walser: 

Excellent. I would just make one comment? the question 
really hinges around the reason why we are having this 
exploratory seminar in the first place — to get some matches, if 
we can, on customer reaction and action requests for quality. 

The next question is, what role do you see private 
career schools playing in the education and training of 
Americans who "missed the boat" as far as public education is 
concerned? 

Mr. Golomski: 

As far as what? 

Mr. Walser: 

As far as public education is concerned, probably from 
the viewpoint of public education. How do you see the private 
schools picking up and dealing with some of these quality issues 
that we heve been talking about, the private, for-profit 
technical schools maybe. 

Mr. Golomski: 

There are a couple of aspects to this. One is the for 
profit schools that you are talking about, bat the other aspect 
is the short term or maybe longer term in-house educational 
programs that most companies have, at least the better 
companies. I see more of the activities in that area, where 
they are relying on expert external help rather than utilizing 
for-profit, external vocational schools or training schools of 
that sort. 

Now, that doesn't mean it couldn't change, but I think 
in some cases some of those schools haven't had the faculty who 
understand the subjects well enough to be able to do the kind of 
job that the three of us have been talking about: to bring 
people up to a higher level of competence. 

Dr. Godfreys 

I don't know that much about the for-profit end of that, 
but as far as the community colleges and technical schools, one 
of the best examples of bringing them in, in a very active role, 
I think, is out on the west coast with Hewlett Packard. They 
started using technical school professors and local high school 
teachers to teach in-house company courses, sharing their 
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materials, their methods, and reviewing and very carefully 
qualifying these people. And then when they became very 
satisfied with the jobs that some of these people were doing, 
then contracted it back to the technical schools to provide the 
training for their work force. And they built a very strong 
partnership that is now carried on, I think, in about its fourth 
or fifth year. 

Mr. Golomski: 

I believe Ford is doing something to this with the 
Schoolcraft Community College, which is somewhere in the 
Livonia-Dearborn area, and anyone who is interested in that 
could talk to Dr. Steve Zayac of the Ford Transmission 
Engineering Department. I am sorry I can t be any more specific 
about it than that, but he spoke at the meeting of the National 
Council of Teachers of Mathematics this last year in Chicago, 
together with a high school teacher who is working on this 
program with him. Ford has allowed a number of the high school 
teachers to sit in on the in-house training programs, and also 
some retired teachers, so they could eventually help with some 
of these activities. 

Mr. Walser: 

This next question has to do with the application of 
quality sciences to the process of education versus exposure of 
individuals during the process of education to the quality 
sciences, and the question is: are individuals on the program 
and the sponsoring organizations in accord as to the meaning of 
National Educational Quality Initiative, which this conference 
is billed as? The presentations make it appear to be a marriage 
of convenience with each partner having an agenda where the 
hidden part of the agenda will doom the effort. 

Now, I hope I did justice to the question — if not, the 
questioner can get to the microphones. 

Mr. Golomski: 

No, I would say that even though we did not rehearse 
and agree what each of us were going to say, we exchanged 
outlines. Although our terminology in some cases might be a 
little different, I think the quality sciences are an emerging 
field also, and so if there might have appeared to be rough 
spots, it is just that we might characterize some of these 
problems in a slightly different way. But I think, in terms of 
substance, we have a lot of agreement. 

Dr. Godfrey: 

I saw no disagreement with what is now the panel, but 
maybe another way of answering the question is, if I had a 
hidden agenda I have also hidden i rom myself. But I was 
struggling with the whole purpose or this conference, and I 
dxdn't get to all of my slides, and the last one I had was the 
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steps that we see in basic quality improvement and developing a 
quality system. It starts with awareness and goes down through 
many of the things, some of the things I talked about, and then 
it gets down to the last ones, which are standardization and 
audits. One reluctance I had to come to this conference, as 
Frank Caplan will agree, was whei s heard that there was very 
much interest in accreditation ana standardization, and we felt 
that we were being invited to work with the American National 
Standards Institute and stuff, rather than work with quality, 
and I thought hat standardizing was not the way to start. 
First we have to understand what we are really doing and why we 
are doing it, and we have to include and we have to have models 
that we really are very proud of, and then try to bring the 
others up to that level rather than, as someone said, I think it 
was Bill, having an accreditation program that guarantees 
mediocrity. 

The other thing that I was struggling with in the whole 
talk was trying to show how some of the ideas that we learn in 
product development were applied in course development and 
course delivery, and at Bell Labs when I was in the Quality 
Theory and Technology Department, we never had training and 
education. That was part of the education and training center. 
And we were very unhappy with some of the things they had done, 
and when we started some of the courses for changing the way 
AT&T made products, we did not even ask the education's 
involvement. Now they have become partners, and now they are 
teaching some of the courses, and co-developing others. They 
had to change many of the things they were doing. 

When we started developing these courses and looked at 
the process we had for course development delivery, it was so 
different from the way we developed the product that we were 
shocked. We had no real good understanding of the users. We 
didn't understand what people did. We didn't understand how to 
get their feedback. We had no reviews of the course as it was 
developed. We had no checkpoints. We didn't have peer reviews 
and we didn't have reviews from the customer. We didn't have a 
project development methodology for curses. All of this was 
developed, and we kept thinking we were playing catch-up, 
because every time we'd look at something we'd say, my gosh, 
we'd never develop a product that way. And half my department 
developed software, and we'd say, we don't even develop software 
that way, and you know how far behind we are in software. 

And we were trying to bring our educational standards up 
to at least our software standards, you know, trying to get 
toward hardware, and we were way behind. And once we started 
making some exciting progress, we found out the education center 
had been doing some very similar things, and we paired off, and 
I think made a lot of progress in applying the same product and 
the progress development methodology into the product of the 
courses and the process of delivering the courses. That is what 
I was trying to show. I think this will work. There is not a 
whole lot of evidence around the world that it does, but there 
are companies other than Bell Labs and AT&T that have been doing 
this. 
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Mr. Walser: 



There is a term in the worker motivation literature 
called discretionary effort and it is that job effort or 
potential effort available after the person has performed 
sufficiently to keep from getting fired. Isn't worker 
performance quality primarily a matter of motivating workers to 
use discretionary effort in their job, and doesn't this seem far 
removed from secondary school education? 

Dr. Godfrey: 

Let me go after this, because this is one I had a fun 
time with on the Wall Street Journal report, not too long ago, 
with Tom Pedricks, the author of Thriving Chaos, bee -se he was 
into the people, people, people is all quality is about. And 
that systems were all unnecessary, and it was really motivation. 
I just had a real problem with it and, you know, we heard Dr. 
Greer start off this morning talking about his son who didn't 
want to strike out, but he didn't want the knowledge and the 
skills and the tools to keep from striking out because that was 
a lot of effort. 

We have American companies just full of highly motivated 
people from the best colleges in the world working until 
midnight every . . ght. We have people averaging 70 and 80 hours 
a week in the Denver Bell Laboratories for years, on some of the 
products that they are competing witt 1 . People are working very, 
very iard. They are trying very, very hard. They often don't 
know what they are doing. They often don't have the skills, 
which is really practice, which is really doing it over and over 
again the right way. 

Bill Conway, who used to be chairman of the Nashua 
Corporation, gives a beautiful example of this. He says you 
take two employees and bring them in the room, and say, we're 
going to have a contest. The prize is a trip for you and your 
wife, you and your husband around the world, all expenses paid, 
first class travel, anything you want to do. 

He says, the game is who can screw the screw in the 
board the fastest. He says, John, here's a screwdriver, go to 
it. Mary, here's a screw, good luck. He said, who is going to 
win? The guy with the screwdriver. He has the tool necessary, 
and he prooably knows how to use it, and all that sort of stuff. 

He says, both of them are highly motivated. Both of 
them really want that trip around the world. But you have got 
to give the people the knowledge, the skills, and the tools. You 
know, my example that parallels Dr. Greer's is golf. I get beat 
all the time by my brother-in-law. I really want to beat him. 
But he plays at least once or twice a week. He takes lessons 
from a pro who happens to be his friend. I play three or four 
times a year, and yet if I am really going to get serious about 
that, I am going to have to learn how to play golf. I am going 
to have to practice, and maybe in some organizations there isn't 
the motivation, but that is not what it is all about. Recently 
I got a brochure in which everybody was concentrating on 
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motivational programs, as if that was the solution, and it isn't. 
It is part of it. So I think this is a multi-faceted problem, 
and to get at single issues I think is a huge mistake. 

Mr. Walser: 

We have about two minutes, and we have four more 
questions, so what I am going to do is read the questions, and 
maybe get a real quick answer on each one with the idea that if 
that is not satisfactory to the questioner, maybe right after 
the session we can still have that interaction. 

Dr. Basili, every analytical comment that you made in 
your discussion of differences between software development and 
production manufacturing models applied directly to K-12 
education explicitly, perhaps to all aspects of education, and 
points up an intensive difference between industrial training 
versus education. 

Dr. Basili: 

I absolutely agree. In fact, I am sorry I didn't make 
that comment myself. And there is a wonderful example. I think 
that what I am saying is that the solution for software is 
harder. But you have to set goals and, of course, you have to 
go down and try and make them operational, and then collect data 
and see if it is working, et cetera. I would argue with 
companies that never collect an ounce of data and just sit down 
and try to set their goals and try to think of what that means 
operationally, that they haven't made enormous strides. I don't 
think we do that now, and I think that is actually even true for 
education. We don't really worry about it, and it is not a 
goal. There is a large number of practical goals. How do you 
set them, and how do you train? Define what they mean in an 
operational way? That is a very, very hard thing to do, but 
even only partly succeeding is a very instructional thing. 

Mr. Walser: 

Next question, Dr. Godfrey. What, in your opinion, is 
needed in the federal government to improve quality planning and 
improvement? 

Dr. Godfrey: 

I hope I will find out next week. As some of you may 
know, the Federal Quality Center is having its first annual 
Federal Quality Conference next week out next to Dulles Airport 
somewhere. I think it is completely booked with a thousand and 
some people coming. They are launching a big exploration on how 
these ideas can be explored widely throughout the federal 
government. Many people have already taken leadership, in the 
DOD, in the Air Force in particular. I mentioned the IRS, BLS . 
Many are already moving, and I think this conference next week 
is to try to explore this, try to focus on some things that can 
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be done, and try to get some kind of resources for the federal 
government employees to make this happen both with the 
management and at the work force level. 

Mr. Walser: 

The next question. You sound like you are familiar with 
the term in education of qualitative "program evaluation." Can 
you compare or contrast quality sciences? 

Mr. Golomski: 

Compared to what? 

Mr. Walser: 

Qualitative program evaluation, a term basically used in 
education, contrast or compare that to quality sciences as you 
have been talking about them today. 

Mr. Golomski: 

Yes, sure. There are two aspects. One is, when I 
talked about quality systems auditing, the parallel is quite a 
bit the same. It is just the criteria that you use for 

evaluation are different, and who does it, and how often is 

different, but in terms of intention they are the same. We 

think the process can be improved by considering some of the 
things we talked about. 

Mr. Walser: 

Thank you. That concludes this section of the 
presentation. I appreciate very much the panel's responsiveness 
in answering the questions that were provided. If you gentlemen 
would vacate your seats, they will be taken by the next group, 
and then we will move on, almost on time. 
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The ASQC has over the last couple of years commissioned 
a Gallup poll on industrial uses of quality control subject 
matter and a number of other questions. This past year, the 
people interviewed were all top management and one of the 
questions was "who is to blame for bad quality?" Over 60% of 
the respondents answered, "The worker in the shop." 

One of the interesting aspects of that is that Bill 
Golomski, Blan, I, Joe Juran, and 57,219 other quality experts 
around the world are quite convinced that the major source of 
problems is not '-he person who is doing the work, not the 
operator in the factory, not the engineer in the design group, 
and not the employee at the lower levels in any government 
agency. 

The problem is management, and fundamentally we say 80 
percent of the problems or 85 percent (the last quote I saw from 
Ed Deming was 94.6 percent, which is a bit more precise than I 
like to go usually) are caused by management. "Management" is 
all those causes that are not attributable to the operator, and 
therefore we are expecting the operator to do what we, as 
management, have failed to do. This is the reason that 
motivational programs by that name are for the birds, because we 
get people all stirred up and strongly motivated and they can't 
do anything. The problems that we are asking them to solve are 
our problems, not theirs. And it is just an impossibility. It 
leads to enormous frustration. Organizations that have gone 
that route have almost universally discovered that they are 
worse off after the program fails, as far as morale of the 
organization is concerned, than they were when they started the 
program. 

The last session, entitled Panel Wrap-Up and Following 
Effort Identification, involves three individuals (four in this 
particular case, because we had to come up with a substitute in 
a hurry a day or two ago). Mr. Spencer Hutchens, who is the 
president-elect (as of July 1st, he will become president of the 
American Society for Quality Control), and Dr. John Burr, who is 
a vice president of the Society, are jointly substituting for 
the current president, Doug Ekings, who is not well and was 
unable to attend this session. 

But we obviously have ASQC's top echelon well 
represented. We have Dr. D. Kay Wright, the Acting Deputy 
Assistant Secretary of the Office of Vocational and Adult 
Education of the Department of Education, who will be 
representing the Department in this discussion, and Ms. Virginia 
Robinson, who is an education writer and editor involved with 
the National School Volunteer Program, very much interested in 
the subject, but more or less an outsider to both aspects of 
this, who is representing that organization on the wrap-up 
panel . 

We will start with three brief presentations, four in 
the case of the double ASQC work, to express what they have 
derived from the discussion we have had so far today, and then 
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we will proceed to talk about what we think can be done 
effectively on a follow-up basis. And with that introduction, I 
would like Dr. Wright to start with the first presentation. 



Dr. Wright: 

Thank you. First of all, it has been a very enjoyable 
and productive day for me. I have learned quite a bit. In 
fact, as I sat there and listened to the presentations I 
thought, I wish I could have a whole course on this. That kind 
of goes with my way of thinking about what we could do as a 
result of meetings such as this, and that would be to see if we 
could get it incorporated into teacher preparation institutions 
and administrative training. I have been a principal at all 
levels. I have been a counselor. 1 have been in education, it 
seems, all of my life, and I have never had an exposure to 
something like today. So I think that it was extremely 
valuable. 

I represent the U.S. Department of Education today. 
However, when I speak to you in more detail in the second part 
of this panel, it will be in relation to vocational and adult 
education, because that is my area of expertise, and also I feel 
that it lends itself very well to the subject today. So I am 
going to pass on at this point and say that it has been very 
productive, and I think that the only Lhing I have to say in 
regards to the particular presentations is that I think you have 
raised the consciousness to a level that it needs to be, and 
from now on we are all going to see who is in that circle, and 
the outreach and the input. 

Mr. Caplan: 

Thank you very much, Dr. Wright. Ms. Robinson, please. 
Ms. Robinson: 

I am going to speak entirely to the issue of elementary 
and secondary education, K through 12. As I listened to +-he 
presentations today, I kept thinkino analogies. Yes, I can see 
how that might be something that could be used, or can it be 
used in the dynamics of the way schools work. They talked about 
scrap and rework, and I thought drop-outs. Compensatory 
education. Do it right the first time. All principles which we 
have generally assumed do not operate in education. You can't 
be sure that even if you do it right the youngsters are going to 
get it right, and some of them are going to have reverse 
motivations to leave school. And so, I was going along today 
and I thought I would like to know, for example, the reaction of 
a quality control expert to a situation which educators 
frequently rely upon to excuse their failures; they have no 
control over the quality of the material with which they are 
working. I think it would surely be a major challenge to 
quality control science to determine how you have achieved 
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quality control in the absence of control of the raw material, 
which in the case of education, of course, is the students who 
come to you with a wide range of provocations. 

That brings me, again, to compensatory education, which 
we now take for granted as part of the elementary and secondary 
scene. Some children will need to go through the cycle more 
than once or in fact need to be separated from their peers and 
taught in a different way. How can one apply quality standards 
to reaching a wide range of youngsters who we have decided we 
will separate out from the process? It would be as if a factory 
had two operations, one for standard materials and one for 
substandard materials. 

I also thought, on the hopeful side, as I listened today 
that I got a little bit closer to feeling there could be a 
breakthrough on the education reform scene if some of these 
concepts could be incorporated. It has seemed fairly depressing 
to those of us who have watched the overall education picture 
since the most critical reports came out five years ago that not 
a heck of a lot had changed. A little here and there, but 
fundamentally very little change. 

Today I got the feeling that maybe, maybe this might do 
it. Maybe this is a key here f something that could happen that 
would be profoundly different, and then it occurred to me again 
that it is very hard to change the dynamics of so huge an 
institution as elementary and secondary education, and I got to 
thinking in more specific terms. 

For example, a profession is characterized, I believe, 
by setting its own standards. The practitioners set the quality 
standards for the profession. Education currently is deep in 
the question of whether teachers are truly a profession. They 
do lack that one essential hallmark of a profession. Teachers 
do not set their own professional standards. They are set for 
them by external bodies, state accreditation, or certification 
units, primarily. 

So that is an issue that I think will be interesting in 
quality science as applied to the professions. What is the 
status of that concept of professional standards and quality 
control? 

I am also moved to observe that it is only very recently 
that in elementary and secondary education outcomes have bepn 
considered a measure of quality. We have used inputs, 
facilities, degreed teachers, certified teachers, and so forth, 
to determine whether it was good and successful. Only very 
recently has anybody talked about the quality of the product, 
and there the general concept arises again with which I opened, 
which is blaming product for its own inadequacies. The 
education system did what it could but the kids didn't have what 
it took, or they didn't work, they weren't motivated, their 
parents didn't care, they watched too much television, they've 
got second jobs after school. You name it. There have been 
many, many, I will use the word excuses, raised by educators for 
the fact that they could not control the quality of their 
product, and I think it would be extremely interesting to have 
quality control people address this head on. 
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I mean can you, indeed, guarantee the quality of a 
product even within the quality control range, which I know is 
never 100 percent? Can you guarantee it, if you cannot control 
the material with which you work? 

I guess one of the most ironic things that I heard was 
the comment that quality assurance may take the form of how to 
do the wrong thing more efficiently. I guess if you were going 
to have an epitaph for the current education reform movement it 
would be that. I think that a lot of energy is being spent on 
how to do the wrong thing more efficiently. 

I just hope that maybe apart from those of you who are 
educators you might be interested in these comments and will 
encourage our quality experts to think about this. It is a big 
problem that needs the best kind of thinking that anybody can 
bring to it, and if you've got anything to contribute I think 
you should do it. 

Thank you. 

Mr. Caplan: 

Thank you very much. Those are very interesting 
comments. We appreciate those. I guess Dr. Burr is going to be 
first up on the quality side, which makes sense, since he has 
been here longest at the meeting. John. 

Dr. Burr: 

About two days ago Frank called me up on the telephone 
and asked me if I would be here today and explained that Doug 
Ekings would be out of town and not feeling too well, and I said 
sure, and I hung up the phone. My daughter said, who was that 
on the phone, and I said, that was Frank Caplan. He asked me to 
substitute or replace, I don't know which, Doug Ekings and she 
said, well, what is the difference between substitution and 
replacement? And I said, well, if you go out into a cabin and 
you break a window, you if you don't have materials right there 
to fix it, some of us will put up a piece of brown paper, and 
put a little X on it, and that is a substitute. Then you can't 
see through that very well. 

Now, another thing you might do is go into town and buy 
yourself a piece of glass and put that in. That would be a 
replacement. And she saiu, I know what you are. I said, what's 
that? She said, you're a replacement. I said, what do you 
mean? She said, you're a pain. 

Mr. Caplan: 

I thought she was going to say she could see through you 

easily. 
Dr. Burr: 

I want to take a few minutes just to sort of summarize 
what I thought were the ultimate key concepts that were 
presented today. The first one I think is very obvious, that 
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systems, aU systems, are processes. T v * processes can be 
improved; that improvement never, ever .3; that change must 
start at the top. Everyone must be involved. Requirements must 
be customer-driven and, finally, education is a system and it is 
a process, and we start back with Number One. 
Thank you. 

Mr. Caplan: 

Thank you, John. The next speaker is Spencer Hutchens 
who, as I indicated to you, is the president-elect of the ASQC, 
and he got the same kind of phone call that John did. 

Mr. Hutchens: 

I certainly did, so since John has done such a fine job 
telling how the telephone call came I don't have to review it 
with you. I might say, though, that I am from Los Angeles, so 
it was a little bit more of a problem for me than it was for 
John to come from Rochester, New York. That is not really the 
reason why I was late getting here this morning. I had another 
meeting scheduled, and I just could not reschedule that one 
totally, so I arrived this afternoon. I was very impressed with 
Dr. Basili's presentation. 

I was very excited when we met two or three months ago 
to discuss the possibility of ASQC being a participant in this 
seminar that is designed to explore ways to improve the quality 
of education. I don't think there is anything in our country 
that concerns me more today than education. Although we don't 
have all the answers, based on observations made by our 
membership, there is a real need for action to improve the 
system at all levels. 

I wonder if our current system can provide the skills 
required to support the level of sophisticated hi-tech programs 
expected in the 21st century; will that generation be prepared 
to deliver the higher skills required? 

So I am very pleased to be working here with all of you 
today. And you will hear from me again a bit later, 

Mr. Caplan: 

Thank you, Spence. Well, that completes the first phase 
of what we want to cover in this section. Now, I have asked Dr. 
Wright to beg ! n the second phase, and of course she told me she 
had some ideas about what she thought could be areas where the 
quality professionals in our society could contribute in working 
closely with the Department of Education. Kay. 

Dr. Wright: 

Okay. Thank you. First of all, let me give you a 
little bit of background. This past year I am sure you are 
aware there were a series of Congressional hearings held on 
global competitiveness of the United States, and over and over 
again, as our speakers said this morning, they pinpointed that 
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the educational system was key to turning around the economy and 
making the United States competitive once again in the global 
market. 

So, with taking into account the hearings and all of the 
reports that have been issued, the Vocational Office came up 
with a series of ideas, I will call them principles, chat we 
think would be important to put into place in each vocational 
program. We are saying these things loud and clear to the 
states and to the directors of vocational education programs, 
and we could probably use all of the assistance that we could 
get in putting these things into practice. Let me go over them 
very briefly for you. 

One is that we want every student to come out of our 
secondary schools with the basic skills, and I am sure that is 
no surprise to anyone. But definition of basic skills would be 
a key area. Everyone talks about it, but what is it? So that 
would be an area that we could use your assistance. 

Secondly, we are telling our vocational program 
directors that if they do offer occupat ionally specific programs 
at the secondary level, and by that I mean job training programs 
that are very specific, such as technician training in 
electronics, welding technicians, aid so forth, that they relate 
directly to the labor market, and that the people in the labor 
market are part of the process for delivering those programs, 
that they help plan the curriculum content, that if necessary 
they collaborate with the schools on the use of equipment, team 
teach with instructors, and so forth. 

The third area that we are asking them to work in is to 
expand partnership efforts with all business and industry. 
Again, it is going beyond the feel-good sorts of partnerships 
thct were established some years back when businesses and 
industries adopted schools and gave them money for special 
projects and for equipment. We really want to blur the lines 
between education and business and industry, and work much more 
closely. 

Fourth, we'd like to see a lot more planning taking 
place at the regional and state levels for all entities that are 
delivering job training. That would be from the secondary 
schools, to the community colleges, to the area vo-techs, to the 
Department of Labor, Department of State, the private, 
proprietary schools that are providing training, to sit down and 
take a look at what is needed in the region a^ far as job 
training, what is available in the region, what is available at 
what cost in the region, and then try and share resources. 

I think we are past the point where we can all go out 
and buy the latest equipment and duplicate efforts, so we are 
asking people to sit down and plan regionally. 

And the last area, ard this is again an area that I 
think would lend itself with working with your association, is 
that we are telling the schools that if they offer 
occupationally specific programs that they should measure the 
student compentency. If a student is being trained as a 
welder, or if the student is being trained as an electronics 
technician, there are skills that should be mastered, and those 
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skills should be demonstrated in performance and also in 
writing. And so we are asking states to take a look at how they 
would do that. 

In Pennsylvania, I just came from Pennsylvania to 
Washington, we worked with SOCAT (the Student Occupational 
Competency Achievement Testing Corporation), to develop tests to 
measure the student competencies in 34 occupational areas. 
Those tests were developed with business and industry people 
working alongside educators to say this is important, these are 
the things they should be able to do when they leave school and 
come in at entry level. So these are some of the things that we 
are thinking about and talking about in vocational education. 
They are some areas that I think lend themselves to working with 
your group and with others to see if we can assist the states as 
they wrestle with these measures of accountability and quality 
that they are being faced with. 

Mr. Caplan: 

Thank you very much, Dr. Wright. Ms. Robinson. 
Ms. Robinson: 

She has just stolen my two key words, which were 
accountability and quality control. I think if I would ask one 
simple thing of the quality control message it would be to think 
about and to provide some philosophical leadership in relating 
this word, accountability, which is either the hope or the 
bugaboo of education right at the moment, depending on where you 
sit, I guess. Certainly the stress is on accountability, and I 
don't see how that can be divorced from quality measures, 
quality control, but there is no external value being given to 
educators on that subject at the moment that I know anything 
about. And in line with the comments that have been made here 
today, it sometimes takes an outside expert. It may well in 
education, I think, take someone from outside the profession of 
administration or teaching to clarify this rerlly serious word, 
accountability, that is either going to be th<> point on which 
educators dig in their heels and say, I am going to go down this 
road and you've got all the accountability you are ever going to 
get, you know, or it may open a whole new way of thinking about 
the education of those children who are not old enough to be 
asked, as your engineering students were asked, what they wanted 
to learn. Somewhere there has goi: to be some relationship 
between controlling quality and accountability and so that would 
be my contribution. 

Mr. Caplan: 

Thank you very much. Spence. 




Mr. Hutchens: 

I am sure that ASQC is a resource, and I ask you to 
please use it. We are interested and we want to work with you. 
Now let me tell you a bit about ASQC. 

ASQC is a non-profit, non-political, non-lobbying 
national society consisting of more than 63,000 members and 400 
corporate members representing ever* facet of industry and many 
government activities. There are 120 sections, 13 divisions and 
12 technical committees. We are proud that among the membership 
are the top professionals in the field, including W. Edwards 
Deming, Jopseph M. Juran, Kaoru Ishikawa, Philip B. Crosby, 
Armand V. Feigenbaum, and the late Walter A. Shew 1 art. Most of 
the quality methodology and management principles now in use 
were originated by these past presidents and honorary members of 
ASQC. 

The divisions and technical committees sponsor meetings 
and special events including training throughout the country. 
Each publishes a newsletter. There is also an International 
Sec -ion that represents those members outside the United States. 
There are about 2000 members in this category. 

Each year since 1948 ASQC has held an Annual Quality 
Congress. It is one of the nation's most respected technical 
conferences. It offers quality professionals, who share a 
common concern for quality, an opportunity to exchange ideas 
with colleagues from around the world. 

In 1984 ASQC initiated * national campaign to capture 
and direct the attention of business and industry to the 
strategic imperative of quality improvement. The initiative 
resulted in a Joint Resolution of Congress and Proclamation by 
the President of the United States that designated October as 
National Quality Month. 

National Quality Month has attracted some of industry's 
top executives to serve as chairman of the Annual KQM Forum 
because they are fully aware of the significant role that 
quality plays in today's competitive global marketplace. The 
late James E. Olson, CEO, AT&T ; Douglas D. Danforth, CEO, 
Westinghouse; and James R Houghton, CEO, Corning Glass; have 
served as chairmen of NQM. Colby H. Chandler, CEO, Kodak, is 
the 1988 chairman. 

Dr. Wright, you expressed an interest in learning 
skills. We do understand the importance in learning basic 
skills. Our divisions and technical committees are designed to 
improve skills in individual technical areas such as aircraft, 
aerospace, chemical, mechanical, electronics, et cetera, for 
people of all ag . 

Accountability was mentioned with quality. Yes, they 
are related. Be it manufacturing or training, the person 
performing is responsible for doing it right the first time. We 
can't forget education of the children. Every effort must be 
made to desi'jn learning programs that provide quality education 
and the motivation for improving the system. ASQC is very 
desirous of working with you in improving the quality of 
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education. We have many resources that are available throughout 
the country. Our headquarters staff is available to help 
identify thv* resources that you need. 

I think that when you talk about accountability being 
related to quality control certainly it would not be too wrong, 
I don't think, to say that quality control is understanding that 
you are accountable for something, that you must make sure that 
it does happen right and, if at all possible, to try to have it 
happen correctly the very first time it is manufactured or the 
first time you do something. I think this might also have 
something to do with education for the children that we are 
talking about in the lower grades, in primary* school, to make 
sure they don't repeat grades, you know, to be sure they are 
learning to do it correctly the first time. 

But I guess one thing I would like to leave with all of 
you today is that ASQC is very willing and we certainly would 
like for you to use the resource that we have. What kind of 
resource do we have to offer? I think that it is a very, very 
broad range of resources that we have and we can certainly work 
with you once this is over to identify some of those resources 
that we may be able to offer and have you make good use of. 

Mr. Caplan: 

I might mention that one of the questions that was 
addressed to me during the break was, what could ASQC do in the 
area of support, what could we provide in the area of texts? 
The American Society for Quality Control has as one of its arms 
Quality Press. We obtain manuscripts from authors. We 
commission books. We publish books. We can provide textbook 
development support in any area and at any level we might 
mutually decide upon. 

It would also be highly appropriate, as far as I am 
concerned, that we become involved in answering que°«-ions of 
curricula revision cooperatively with the educational community. 

As you recall, Bill Golomski gave seme of hi. ( thoughts 
as to what level of instruction should be incorporated at 
different grade levels, and while there may be a variety of 
ideas as far as that table is concerned, nonetheless it is that 
sort of thing that we could very rapidly help with, and as that 
form or format was identified, we could then commission the 
development of suitable texts in that area. 

Now, these might well be developed by some of our 
members who are already educators in the school systems 
developing curricula materials for other programs, so that we 
would not be doiny this entirely as an outsider in that sense. 
We would be dealing with the fact that I would estimate at least 
30 percent of our membership, totaling to Spence's figures 
15,000 or 16,000 people, are involved with education and 
training in various ways, and this means that we have a rather 
considerable resource of people who do instruct in a variety of 
circumstances. Most of those circumstances are at the junior 
college, community college level, A number of us are involved 
in full four-year college and graduate programs. But there is 
also some small amount, but not very much yet, of the kind of 
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work that Spence mentioned is going on at Corning, where we are 
actually working with the secondary schools to accomplish this 
sort of thing. 

We can help and we would very much like to help. 
Actually, we would be a little bit distressed if you went off 
and did this without asking us to help, because it would be just 
like we were talking about earlier. Please identify who your 
customer is and your supplier, and let's all work together to 
make this product come out the way it ought to. 

LeRoy? 

Mr. Walser: 

One thing that all today hasn't really been mentioned is 
that we have been in motion, Spencer, that ASQC is the 
accredited, sponsoring organization for Committee Zl on Quality 
Assurance for the American National Standards Institute. I 
think that's just the work that might be a real opening, and I 
know that Blan indicated that it is a little premature to talk 
about standards from that perspective. I would like to broach 
that topic just briefly and say that forum itself, even though 
Zl has a lot of divisions and activities under it, might be an 
exploratory forum for getting together what the topics of some 
of these curriculum items might be at different levels of 
education. That would be a possibility. Am I off base, or is 
that related? 

Mr. Caplan: 

Not at all. As a matter of fact, Bill, Blan, and I are 
all members of the Zl Committee and it should be, I think, 
highly appropriate for the committee to address this question as 
a special task. It is not exactly what we would ordinarily 
address, but I think it would be something that we could deal 
with. Any feelings, Bill, Blan? 

Okay. The answer is, yes, I am sure we can do it. You 
must recognize that the Zl Committee involves somewhat more 
than 100 members, most of whom are not ASQC members, because the 
Zl Committee was formed from a large number of organizations who 
have an interest in quality, including a number of government 
agencies, so that there is a tie-in there that might not be 
obvious . 

Mr. Walser: 

I think with that tie-in might also be necessary to 
broaden the customer base or the involvement of other folks who 
would have a need to have a say in what the outcome would be in 
education. I think that was also Virginia Robinson's point 
about how do we get a definition of some of these quality, 
accountability factors and where do they fit. 




Mr. McKee: 



I am Sam McKee with the Department of Education. I 
think the ASQC should pursue the idea that we were just talking 
about, but I think the major contribution that can be made to 
education is this process of quality control, quality assurance; 
this is a coordinated process. The customer, it seems to me, is 
the 15,000 school districts in the United States. Now, that is 
quite a proposition, to reach all 15,000 of them and persuade 
the school boards and the superintendents (in order to get 
started with their top down management) to adopt that process, 
but that seems to me to open a door. It seems to me that a 
combination of help from the Department of Education, the 
appropriate associations, and the expertise you have about this 
process, is the way to reach the customers and really make an 
impact on education. 

Mr. Caplan: 

Thank you. That is an excellent point. Spence. 
Mr. Hutchens: 

I agree that that is perhaps an area where we can be of 
the most help in getting the process started. I also agree thai: 
there is a need to identify parameters that we use in quality 
assurance and quality control that are related to education. We 
could help develop procedures to control the quality of 
education at various levels. We would certainly be happy to 
work with you. 

Mr. Caplan: 

Thank you. John. 

Dr. Burr: 

Yes, you recall the old story about when you are up to 
your neck in alligators, it i3 hard to remember that your job 
was to drain the swamp. The question you have to deal with in a 
lot of cases is that there's a lot of problems and a lot of 
pressures and stress. And you must try to come in and persuade 
the top person, in the face of that swamp full of alligators, 
that this is what must be done. And he or she is looking at 
that just an iterative position system, and somehow or other 
while the enlightenment that may come from outside the 
organization, I think you are going to find that the impetus for 
that is going to have tc come from a higher authority, such as 
the New York State Regency Board, or the Department of Education 
at the national level, or wherever, but I think that is where 
the encouragement is coming from. That change is going to have 
to come because the people who are going to be participating in 
that change are uncomfortable with where they are and, in many 
cases, I see they are very comfortable where they are now. 
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Mr. Caplan: 

v 

I think that is an excellent point, John, and it is 
something that I had intended to refer to in response to your 
comment. I have a very specific and somewhat bitter experience 
personally with attempting to do exactly what you suggested, not 
now, not in this case at the local school board level. , but at 
the State University of New York area combined with the 
Department of Education. New York state, for those of you who 
don't know it, has two separate Departments of Education. One 
is called the Department of Education, which \s K through 12. 

I got these people together, and we had discussions, we 
had a very, very nice seminar, and we all agreed it was the 
greatest thing since sliced bread, but for some reason or 
another it never got carried on. What I then decided was to 
write a letter to Secretary Bennett saying, look, my friend, the 
Honorable Dr. Bennett, et cetera, et cetera, we have a problem 
in this country and we can't get the schools interested, and we 
need help. And by the way, it was not just my experience in New 
York, but it is all over the country, with local exceptions, and 
very, very great ones, but there are very few. We need help. 
We need some pressure applied. At the college level we have a 
mechanism, the accreditation boards. And we can deal with them. 
Dr. Reyes-Guerra, from ABET, was here a little bit ago and 
thoroughly endorses what we are doing, and we are getting 
excellent support from COPA and so forth. We are making strides 
at the college level. We are at a complete stymie when it comes 
to the primary and secondary schools, as far as being able to 
address it, because there's 50 different accreditation groups. 
We don't know how to go about doing this. There is no formal 
national accreditation function. It is handled by the states. 
And we are in the soup. 

We felt that by taking this initiative and getting this 
seminar set up we could make some beginnings towards getting 
this done, but in all honesty we tried the route you are talking 
about. It is possible, but it has limited success and then only 
in certain areas and, very frankly, there is enough change-over 
in school boards and so forth that you have to go back in and 
re-sell the program every three or four years. And we need to 
have it built-in so that there is not squirming out from under 
or just ignoring it completely when the next budget crunch 
comes. 

Dr. Wright: 

I agree with what you just said. I would suggest, 
however, that we work closely with the state departments of 
education because, as you indicated earlier, each state has its 
own accreditation standards and so does the state board of 
education, but they are all concerned with this issue of 
quality. And they will be concerned more in the future with the 
issue of accountability; the federal government was making it an 
issue and wanted to know how to develop measures to demonstrate 
that they are meeting the accountability issue, so I think that 
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we can work cooperatively with the states. And with the federal 
position on accountability being stated the way it is, I think 
some of these things will come about. 

Mr. Caplan: 

I appreciate that comment, Dr. Wright, and with the 
exception of Alaska, we have key members in state capitals in 
all the other states of the country, and we certainly would be 
more than willing to try to motivate that activity. On the 
other hand, unless there is some rather clear national directive 
towards quality as separate but equal to accountability, we are 
not going to achieve it even by that method because we have 
people with different skills, different knowledge in these state 
locations. We are not organized to put on that kind of drive. 
We can address it and will address it, but it is not going to 
happen without some very strong assistance from the Department 
of Education. 

Ms. Robinson: 

That may be true. It may be that the state is the best 
way to go, but I would offer you an alternative suggestion. No 
one is more uptight about accountability than the classroom 
teacher. I would urge you to take your concept to the major 
teacher organizations. 

Mr. Caplan: 

They were invited to be here today. 

Ms. Robinson: 

Were they? Contrary to the principle which was 
enunciated this morning that quality control has to be top down, 
with all respect, that isn't the way education works. Education 
works in the other direction. You cannot achieve anything 
unless you first convince the classroom teacher of its value. 
Quality control strikes me, just off-hand, as something the 
classroom teacher would find interesting. I really think so. 
Accountability, evaluation by the principal, those are bad; but 
something that a teacher would see, some strategies to employ in 
their classroom that would then be reflected in the evaluation 
that was made of their teaching, this might be an alternative 
route to go. 

Honestly, state departments of education and, even with 
its superior clout, the U.S. Department of Education have 
limited capacity to influence what actually happens in the 
teaching process, which is, after all, what you are getting at, 
isn't it? 
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Mr. Caplan: 



Yes, it is, and let me point out to you that what 
happened in New York state with the Board of Regents initiative 
was extremely revealing to me. 

Ms. Robinson: 

Did they make a difference? 

Mr. Caplan: 

The local school districts went ape. They appointed 
teacher groups to work over the summer to develop curricula to 
cover these materials in the classroom. 

Ms. Robinson: 

Well, the regent is a special thing. The regents' 
examination is unique. There are states, however, that do have 
some punitive measures built into their accreditation process, 
and that might be a similar tool, but I still say that isn't 
going to make any difference to what actually happens unless you 
get the practicing teachers to buy into the idea. I am also 
saying that I think it sounds like an attractive alternative to 
everything we have got now. 

Mr. Caplan: 

Okay, we will work that angle too. 
You had a question? 

Mr. Scruggs: 

I think we are making one of the worst mistakes that we 
can make. We are here for a session on quality. One of the 
things we are supposed to do, one of the first things, is to 
identify the customer. The last ten minutes of the discussion 
have oeen within the confines of the education community. When 
you are talking about customers of educational products, the 
students, you are talking about business, you are talking about 
universities; when you are talking about the quality of the 
public schools. You have business expectations, business 
customers. You have university and college expectations. You 
have military. All these organizations take the products of the 
school system, so you need to talk to these people. 

You also need to talk to the parents, the PTA 
organizations, for example. All these people need to be drawn 
in to establish quality standards for the schools. There are a 
couple of practical ways that the top leadership of education in 
the country, the Secretary, the Under Secretaries, and the 
Deputy for Intergovernmental and Interagency Affairs, that would 
affect not what we are talking about throughout the country, but 
within the Department of Education. As a result of this meeting 
the books that were published at James Madison High School could 
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be re-examined to see whether they were specific enough in terms 
of student behavior. That is what we were just talking about. 
Maybe it should be made more specific, drawing upon the various 
customers of our educational system. 

Another thing that the Department of Education could do 
is to apply the content to the management of its program. There 
is something that just came out of the Office of Management and 
Budget called OMB Circular A-132, Federal Quality and 
Productivity improvement in Service Delivery. It sets forth all 
these things we have been talking about today, beginning with 
the identification of the customer and establishing standards of 
quality and productivity and timeliness and setting annual 
incremental goals for improving in each of these areas. 

On May 24th, day before yesterday, the Under Secretary 
send out a memorandum to all senior officers indicating that one 
of his key concerns is productivity. Now, it is very nice for 
the Under Secretary to do this, to send out this memorandum to 
the senior officers. I suggest that Secretary Bennett and the 
Under Secretaries and the Deputy Under Secretary for 
Intergovernmental and Interagency Affairs bring together all the 
senior officers of the Department of Education and meet with 
them to show in a more concrete manner that they really think 
this is important; or important enough to merit more than just 
sending out a memorandum. And then I think that the 
requirements of this circular should be implemented within the 
Department and see whether we can do it. If we can do it in the 
Department, it would make it a lot earier to do it out in the 
real education world. 

Mr. Ullman: 

I'm Neil Ullman. There are so many of the areas I would 
love to make comments on, but one of the things is, you know, 
you have to talk about education still as a process, and we are 
talking about some of the intermediate stages. It is true what 
we want to have at the end of this time. I am reminded in 
the last couple of comment of Charlie Schultz's recent Peanuts, 
and the question of fractions now, and she is absolutely up 
tight and can't remember whether she got on or off the bus to 
go to school. 

At the same time I am seeing things that say, should we 
teach reading? There was an article in the New York Times this 
Sunday on should we be teaching reading to pre-kindergarten 
because there is this high pressure to get children to learn how 
to read at a very, very early age. Now, we don't have the 
information. One of the things that has been tried, I think at 
the same time we were trying all these activities, was 
behavorial objectives. It was the thing, a few years back, that 
we measured things and you could see what the outcome was, 
presumably. We don't really know when is the best time to put 
our efforts into the education of the children, and this has so 
much of an impact on the quality. I think we have to be able to 
get in there and figure out really what the goals are in many 
ways before we start thinking about testing, know what the basic 
skills are. I have a calculator that can do fractions, long 
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division. It cost me $10. It is made by Casio. We spend a 
whole year in education teaching how to do some kind of 
algorithm, but we don't need to teach this any more because we 
don't think we need them. I think ASQC can help the education 
community perhaps in trying to help define the way in which to 
go about measuring, the way we should go about setting goals and 
objectives as a first step in defining what the needs are. 

On this issue of accountability I think we are way too 
premature. I think a lot of stuff we talked about in this word 
"accountability" on teachers or on how it is really destructive 
to the whole system. I will stop at that point. 

Ms. Malloy: 

My name is Lynn Malloy and I am with the Nuclear 
Regulatory Commission. I feel a little out of place in this 
area because I feel I really don't have a very large 
appreciation for the type of problems educators are dealing with 
these days. It has been a number of years since I have been in 
school and somehow I feel the process served me very well, as it 
turns out. I think it was Neil that was just talking; I wanted 
to say a little bit on his comment. 

I feel a little bit confused in listening to all of you 
talk because for most of my career outside college I have been a 
quality assurance specialist, a QA practitioner, mainly in the 
manufacturing field, and what I see a lack of here is that I 
really don't know what is the problem that you folks are hoping 
to solve. We have to identify what that problem is or problems 
are. And until you can come to grasp with that, you have to 
then identify what you think the causes arc for those problems 
before you can even begin to take any kind of corrective action, 
which is what you are talking about, a systematic approach to 
solving whatever the problem is. 

I think you have more complex of a problem than you 
realize you are dealing with. Maybe you do realize that you are 
dealing with it, but you are trying to nick away at one little 
part of it, and it really is something that is a lot bigger, I 
think. What I have been getting is that we are talking about 
the economic situation in the United States, and I think it is a 
little foolhardy to think that the educational system is to 
blame for the state of the economics of the nation because there 
are many other factors that play into that situation. The 
problem is bigger than just education. But even just to fix 
education you have to know what problems you are dealing with. 
Identify those, think what the causes of the problems are, and 
go after those causes for correction. And you need to know 
where the power lies in correcting those causes because, 
contrary to what someone said, I truly believe you are never 
going to get any corrections unless you can get the level where 
that power exists to realize that there is a problem and take 
the action to make sure that the problems are corrected. 

And it has to c >me from the top down if you are ever 
going to reach some state of correction, and it is not going to 
be something that you are going to be able to do probably in a 



year or two. You have got to set some goals as to what kind of 
progress you want to achieve, and how soon. Set yourself some 
goals and follow it through. 

Anyway, that is all I wanted to say. I wanted to bring 
it down to a more basic level, because I really feel like we are 
not hitting the grass roots stuff here, and that's my opinion 
after 13 years in the field of quality assurance. 

Mr. Caplan: 

That will be taken into account. Let me point out that 
we did define the problem. We defined it in a somewhat more 
limited fashion than calling it the state of the U.S. economy. 
But the way we defined the problem led us to this conclusion: 
the entering work force coming cut of the high schools, coming 
out of the colleges, are uninformed in the field of quality; 
that industry, business, government wishing to accomplish 
anything in that area has to invest inordinate sums in training 
people after they have been hired and are not able to get to 
many of them until after they have produced a large amount of 
whatever it is they are supposed to do that turns out to be 
unsatisfactory. That is the problem as we defined it. 

This led us to the educational solution. Since the 
entering work force tends to come to us out of the educational 
system, we thought we could address this lack of understanding 
of quality rnd the quality sciences, quality awareness through 
the educational process. 

Mr. Hutchens: 

I would just like to make one comment, Ms. Malloy. You 
certainly proved to me that you have done a fine job and learned 
the lesson very well as a quality practitioner. I think you 
describe the process very well. Certainly we must identify the 
problem and, as we have found in our society as quality 
professionals, yes, you must have the attention and complete 
support of top management to solve the problem. I also agree, 
of course, with the people who have spoken to the fact that you 
need to work with the teachers because that is where the process 
will be implemented. The first step is identifying the problem 
and then identifying the cause and developing a solution. 
Certainly in the six hours we spent here today, we aren't going 
to complete the whole process. It is a start and we must work 
together to implement the process that will lead to solutions. 

Mr. Caplan: 

Thank you, Spence. 

In the interest of attempting to meet at least one of 
our commitments when we invited you to the seminar, I herewith 
declare this meeting complete and finished, and thank you for 
your attendance. However, there appears to be a post-meeting 
question. You may leave if you like. 
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Mr. Kunkel: 



This is maybe a topic of closing anyway. My name is 
Richard Kunkel, and I am Executive Director of the National 
Council for the Accreditation of Teacher Education, and we are 
an agency who has been doing a lot about teaching. I want to 
thank the Department of Education and others for bringing this 
meeting and these kind of resources together because I see the 
real large system of resources you can look at. In my group I 
get involved with teachers, principals, AFT, NEA, school boards, 
volunteers, all kinds of other people. But here, there is a 
content in the process that you folks are dealing with that 
really starts with software. I think this whole focus on things 
like ANSI and ASQC and the focus on your kinds of operations, 
how you go about standard setting, can really 1 elp a lot. We 
are working that process already, so don't just look at it to 
immediately prepare great teachers. Give us a chance to get a 
system application, and I think there is a lot of good we can do 
together . 

Mr. Caplan: 

This meeting stands adjourned. Thank you for attending. 



